[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:32 dist-test-slave-21b9 krb5kdc[31823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:32 dist-test-slave-21b9 krb5kdc[31823](info): set up 2 sockets
Aug 09 16:34:32 dist-test-slave-21b9 krb5kdc[31823](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 16:34:33.799544 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.384s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:33 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:33.916108 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38225
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:34.000773 31839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:34.000981 31839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:34.001003 31839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:34.002126 31839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:34.002170 31839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:34.002183 31839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:34.002194 31839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:34.002205 31839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:34.003840 31839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:34.004086 31839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:34.004299 31839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:34.009891 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.009909 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.009968 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:34.010221 31839 server_base.cc:1034] running on GCE node
I20260809 16:34:34.010566 31839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:34.010890 31839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:34.014123 31839 hybrid_clock.cc:648] HybridClock initialized: now 1786293274014073 us; error 62 us; skew 500 ppm
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:34.017762 31839 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:34.018268 31839 webserver.cc:460] Webserver started at http://127.31.15.254:40035/ using document root <none> and password file <none>
I20260809 16:34:34.018466 31839 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:34.018505 31839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:34.018608 31839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:34.019668 31839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "e1a769f3e0d94325bfe95597d6561ac9"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "8e7de8ce66e9a1de71bb800248c29856"
server_key_iv: "e0f308363622ba61fa8aea8ee1f739e5"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.019973 31839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "e1a769f3e0d94325bfe95597d6561ac9"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "8e7de8ce66e9a1de71bb800248c29856"
server_key_iv: "e0f308363622ba61fa8aea8ee1f739e5"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.021456 31839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:34.022266 31863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.022460 31839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:34.022570 31839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "e1a769f3e0d94325bfe95597d6561ac9"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "8e7de8ce66e9a1de71bb800248c29856"
server_key_iv: "e0f308363622ba61fa8aea8ee1f739e5"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.022641 31839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:34.032994 31839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:34.034783 31839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:34.034905 31839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:34.044798 31839 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38225
I20260809 16:34:34.045197 31839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:34.045249 31945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38225 every 8 connection(s)
I20260809 16:34:34.046069 31947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:34.049718 31947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: Bootstrap starting.
I20260809 16:34:34.050992 31947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:34.051004 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31839
I20260809 16:34:34.051106 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:34.051801 31807 external_mini_cluster.cc:1428] Setting key a457c2e44cc38bf45b91aa2862e8b27c
I20260809 16:34:34.051999 31947 log.cc:824] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:34.053108 31947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: No bootstrap required, opened a new log
I20260809 16:34:34.056448 31947 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } }
I20260809 16:34:34.056607 31947 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:34.056630 31947 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1a769f3e0d94325bfe95597d6561ac9, State: Initialized, Role: FOLLOWER
I20260809 16:34:34.056738 31947 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } }
I20260809 16:34:34.056885 31947 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:34.056941 31947 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:34.056989 31947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:34.058058 31947 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } }
I20260809 16:34:34.058173 31947 leader_election.cc:302] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1a769f3e0d94325bfe95597d6561ac9; no voters:
I20260809 16:34:34.058344 31947 leader_election.cc:288] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:34.058375 31957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:34.058565 31957 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [term 1 LEADER]: Becoming Leader. State: Replica: e1a769f3e0d94325bfe95597d6561ac9, State: Running, Role: LEADER
I20260809 16:34:34.058609 31947 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:34.058740 31957 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } }
I20260809 16:34:34.059343 31960 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1a769f3e0d94325bfe95597d6561ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } } }
I20260809 16:34:34.059501 31960 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:34.060024 31959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1a769f3e0d94325bfe95597d6561ac9. Latest consensus state: current_term: 1 leader_uuid: "e1a769f3e0d94325bfe95597d6561ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1a769f3e0d94325bfe95597d6561ac9" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38225 } } }
I20260809 16:34:34.060243 31959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9 [sys.catalog]: This master's current role is: LEADER
W20260809 16:34:34.060477 31981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:34.060518 31981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:34.061693 31973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:34.061880 31973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:34.063479 31973 catalog_manager.cc:1347] Generated new cluster ID: 635de563790443e4b52c3b4f3731851f
I20260809 16:34:34.063536 31973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:34.070446 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.052721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.31.15.254:38225)
0809 16:34:34.052922 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.052930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.053239 (+ 309us) server_negotiation.cc:407] Connection header received
0809 16:34:34.056418 (+ 3179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.056427 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.056541 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.056753 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.058850 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.060223 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.061678 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.062171 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.062287 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.066484 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.066489 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.066489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.066501 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.068519 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.068949 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.068953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.068953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.069079 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.069328 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.069329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.069330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.069636 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.069787 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.070029 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.070171 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":62,"threads_started":1}
I20260809 16:34:34.076694 31973 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:34.077397 31973 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:34.082986 31973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1a769f3e0d94325bfe95597d6561ac9: Generated new TSK 0
I20260809 16:34:34.083155 31973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:34.109184 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:34.196794 31989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:34.196996 31989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:34.197019 31989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:34.198127 31989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:34.198167 31989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:34.198195 31989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:34.199656 31989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:34.199874 31989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:34.200083 31989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:34.204864 31996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.204872 31995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:34.204910 31989 server_base.cc:1034] running on GCE node
W20260809 16:34:34.204872 32000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:34.205331 31989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:34.205596 31989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:34.206751 31989 hybrid_clock.cc:648] HybridClock initialized: now 1786293274206737 us; error 32 us; skew 500 ppm
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:34.209298 31989 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:34.209686 31989 webserver.cc:460] Webserver started at http://127.31.15.193:40401/ using document root <none> and password file <none>
I20260809 16:34:34.209853 31989 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:34.209892 31989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:34.209986 31989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:34.210927 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "e8a726c82fcf42adbe05e930120901c2"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "6ff18c845a0030b3aab10d211171e8a4"
server_key_iv: "d473be99905af9492ef1d2bccdc94db8"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.211225 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "e8a726c82fcf42adbe05e930120901c2"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "6ff18c845a0030b3aab10d211171e8a4"
server_key_iv: "d473be99905af9492ef1d2bccdc94db8"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.212574 31989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:34.213276 32012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.213459 31989 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:34.213549 31989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "e8a726c82fcf42adbe05e930120901c2"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "6ff18c845a0030b3aab10d211171e8a4"
server_key_iv: "d473be99905af9492ef1d2bccdc94db8"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.213603 31989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:34.222393 31989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:34.224246 31989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:34.224354 31989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:34.224531 31989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:34.224777 31989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:34.224807 31989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.224826 31989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:34.224843 31989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.230734 31989 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45065
I20260809 16:34:34.230796 32193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45065 every 8 connection(s)
I20260809 16:34:34.231071 31989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:34.235996 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31989
I20260809 16:34:34.236086 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:34.236229 31807 external_mini_cluster.cc:1428] Setting key 45dba6ae702a1a99809b270b3b5bc28e
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:34.240558 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.231902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:58481 (local address 127.31.15.254:38225)
0809 16:34:34.231967 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.231969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.232150 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:34:34.232612 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.232613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.232625 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.232670 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.233673 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.234165 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.235089 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.235310 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.235369 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.237842 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.237843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.237843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.237850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.239100 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.239607 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.239608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.239608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.239670 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.239930 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.239931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.239931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.240039 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.240086 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.240406 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.240493 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:34.240718 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.231882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38225 (local address 127.31.15.193:58481)
0809 16:34:34.232039 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:34.232075 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:34.232536 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:34.232727 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:34.232730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:34.233400 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:34.233592 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.233597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.234241 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.234242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.235021 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.235024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.235376 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.235377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.235476 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.235950 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:34.235959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:34.237758 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:34.239182 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.239190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.239192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.239514 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.239739 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.239739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.239740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.239869 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.240092 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:34.240098 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:34.240247 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 16:34:34.240357 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:34:34.240511 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260809 16:34:34.241258 32195 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38225
I20260809 16:34:34.241360 32195 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:34.241551 32195 heartbeater.cc:511] Master 127.31.15.254:38225 requested a full tablet report, sending...
I20260809 16:34:34.242071 31888 ts_manager.cc:194] Registered new tserver with Master: e8a726c82fcf42adbe05e930120901c2 (127.31.15.193:45065)
I20260809 16:34:34.242745 31888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:58481
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:34.262009 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:34.342229 32204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:34.342422 32204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:34.342443 32204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:34.343570 32204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:34.343611 32204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:34.343639 32204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:34.345064 32204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:34.345275 32204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:34.345479 32204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:34.350188 32210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.350322 32214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.350199 32211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:34.350427 32204 server_base.cc:1034] running on GCE node
I20260809 16:34:34.350628 32204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:34.350853 32204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:34.351990 32204 hybrid_clock.cc:648] HybridClock initialized: now 1786293274351972 us; error 39 us; skew 500 ppm
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:34.354331 32204 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:34.354714 32204 webserver.cc:460] Webserver started at http://127.31.15.194:33221/ using document root <none> and password file <none>
I20260809 16:34:34.354920 32204 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:34.354969 32204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:34.355100 32204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:34.356171 32204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "004737b4952545249798ee54ed167032"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5474b8f399dae8e8e921f94a472cbd61"
server_key_iv: "7910ec688082da811b17753b53fdff7c"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.356494 32204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "004737b4952545249798ee54ed167032"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5474b8f399dae8e8e921f94a472cbd61"
server_key_iv: "7910ec688082da811b17753b53fdff7c"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.357786 32204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:34.358438 32223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.358583 32204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:34.358666 32204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "004737b4952545249798ee54ed167032"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5474b8f399dae8e8e921f94a472cbd61"
server_key_iv: "7910ec688082da811b17753b53fdff7c"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.358745 32204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:34.371241 32204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:34.373035 32204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:34.373164 32204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:34.373365 32204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:34.373654 32204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:34.373687 32204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.373715 32204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:34.373732 32204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.379844 32204 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:40217
I20260809 16:34:34.379880 32415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:40217 every 8 connection(s)
I20260809 16:34:34.380177 32204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:34.386397 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32204
I20260809 16:34:34.386502 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:34.386684 31807 external_mini_cluster.cc:1428] Setting key 7e5e92d9b3f0c2c2c30bd3606d06974b
I20260809 16:34:34.389434 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.381000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:46249 (local address 127.31.15.254:38225)
0809 16:34:34.381080 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.381083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.381220 (+ 137us) server_negotiation.cc:407] Connection header received
0809 16:34:34.381704 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.381705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.381718 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.381775 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.382692 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.383189 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.384188 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.384381 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.384447 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.386521 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.386523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.386523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.386532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.388139 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.388506 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.388507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.388507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.388570 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.388814 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.388815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.388815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.388936 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.389004 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.389258 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.389345 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:34:34.389590 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.380932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38225 (local address 127.31.15.194:46249)
0809 16:34:34.381110 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:34.381146 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:34.381597 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:34.381839 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:34.381842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:34.382401 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:34.382592 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.382597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.383300 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.383301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.384102 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.384104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.384479 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.384480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.384598 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.385047 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:34.385055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:34.386413 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:34.388226 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.388232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.388234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.388441 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.388624 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.388624 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.388625 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.388753 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.388994 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:34.388998 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:34.389109 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:34:34.389209 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 16:34:34.389359 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260809 16:34:34.390022 32416 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38225
I20260809 16:34:34.390108 32416 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:34.390269 32416 heartbeater.cc:511] Master 127.31.15.254:38225 requested a full tablet report, sending...
I20260809 16:34:34.390622 31888 ts_manager.cc:194] Registered new tserver with Master: 004737b4952545249798ee54ed167032 (127.31.15.194:40217)
I20260809 16:34:34.390985 31888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:46249
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:34.411064 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:34.493036 32423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:34.493244 32423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:34.493268 32423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:34.494390 32423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:34.494433 32423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:34.494462 32423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:34.495968 32423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:34.496196 32423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:34.496418 32423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:34.501232 32435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.501307 32431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:34.501232 32429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:34.501624 32423 server_base.cc:1034] running on GCE node
I20260809 16:34:34.501822 32423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:34.502048 32423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:34.503190 32423 hybrid_clock.cc:648] HybridClock initialized: now 1786293274503170 us; error 39 us; skew 500 ppm
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:34.505880 32423 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:34.506336 32423 webserver.cc:460] Webserver started at http://127.31.15.195:37573/ using document root <none> and password file <none>
I20260809 16:34:34.506548 32423 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:34.506598 32423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:34.506726 32423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:34.507829 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "fb385171a8774c3bb0befa331a006983"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5e59150c32edde577dddf8f0dc6c9f6f"
server_key_iv: "3653cf0b32042efe915b6231c0a4b5ec"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.508170 32423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "fb385171a8774c3bb0befa331a006983"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5e59150c32edde577dddf8f0dc6c9f6f"
server_key_iv: "3653cf0b32042efe915b6231c0a4b5ec"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.509480 32423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:34.510179 32453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.510344 32423 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:34.510430 32423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "fb385171a8774c3bb0befa331a006983"
format_stamp: "Formatted at 2026-08-09 16:34:34 on dist-test-slave-21b9"
server_key: "5e59150c32edde577dddf8f0dc6c9f6f"
server_key_iv: "3653cf0b32042efe915b6231c0a4b5ec"
server_key_version: "encryptionkey@0"
I20260809 16:34:34.510488 32423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:34.534862 32423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:34.537117 32423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:34.537263 32423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:34.537503 32423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:34.537822 32423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:34.537858 32423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.537889 32423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:34.537912 32423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:34.543776 32423 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:46601
I20260809 16:34:34.543833 32634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:46601 every 8 connection(s)
I20260809 16:34:34.544142 32423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:34.545939 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32423
I20260809 16:34:34.546057 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:34.546247 31807 external_mini_cluster.cc:1428] Setting key 74733f2618c7f47d57f7d2daf646b545
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:34.553539 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.544825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:38705 (local address 127.31.15.254:38225)
0809 16:34:34.544886 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.544888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.545110 (+ 222us) server_negotiation.cc:407] Connection header received
0809 16:34:34.545596 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.545597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.545606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.545661 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.546937 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.547655 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.548604 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.548785 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.548836 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.550781 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.550782 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.550783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.550789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.552122 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.552597 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.552598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.552599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.552664 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.552985 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.552986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.552986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.553123 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.553189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.553409 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.553472 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:34.553745 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.544805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38225 (local address 127.31.15.195:38705)
0809 16:34:34.544984 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:34.545028 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:34.545517 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:34.545719 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:34.545722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:34.546520 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:34.546806 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.546813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.547735 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.547736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.548531 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:34.548533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.548843 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.548844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:34.548942 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.549399 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:34.549407 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:34.550703 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:34.552204 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.552212 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.552214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.552508 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.552747 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:34.552748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:34.552748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:34.552912 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:34.553186 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:34.553191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:34.553307 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:34:34.553392 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:34:34.553512 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 16:34:34.554169 32636 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38225
I20260809 16:34:34.554265 32636 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:34.554441 32636 heartbeater.cc:511] Master 127.31.15.254:38225 requested a full tablet report, sending...
I20260809 16:34:34.554813 31888 ts_manager.cc:194] Registered new tserver with Master: fb385171a8774c3bb0befa331a006983 (127.31.15.195:46601)
I20260809 16:34:34.555173 31888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:38705
I20260809 16:34:34.558230 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:34.581350 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.574583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.31.15.193:45065)
0809 16:34:34.574745 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.574749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.574760 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:34.574787 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.574789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.574802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.574874 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.575350 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.576067 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.576722 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.577022 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.577079 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.579114 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.579124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.579124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.579131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.580454 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.580752 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.580754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.580755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.580806 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.580932 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.580932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.580933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.581051 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.581163 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.581197 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.581255 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
W20260809 16:34:34.581728 32063 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58716
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[31823](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:34.603451 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:34.597194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.31.15.193:45065)
0809 16:34:34.597242 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:34.597244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:34.597253 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:34.597318 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:34.597319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:34.597330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:34.597383 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:34.597875 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.598368 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.599031 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:34.599226 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:34.599298 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:34.601395 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:34.601397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:34.601397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:34.601403 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:34.602539 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.602840 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.602841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.602842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.602911 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:34.603044 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:34.603044 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:34.603045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:34.603130 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:34.603213 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:34.603297 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 16:34:34.603372 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:34:34.603976 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31989
I20260809 16:34:34.613385 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32204
I20260809 16:34:34.627992 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32423
I20260809 16:34:34.635521 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31839
2026-08-09T16:34:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2248 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[32660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[32660](info): set up 2 sockets
Aug 09 16:34:34 dist-test-slave-21b9 krb5kdc[32660](info): commencing operation
krb5kdc: starting...
W20260809 16:34:35.778152 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:35 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:35.889966 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38647
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:35.969879 32676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:35.970084 32676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:35.970112 32676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:35.971227 32676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:35.971287 32676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:35.971309 32676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:35.971325 32676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:35.971349 32676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:35.972802 32676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:35.973042 32676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:35.973249 32676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:35.977802 32683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:35.977901 32676 server_base.cc:1034] running on GCE node
W20260809 16:34:35.977824 32682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:35.977802 32686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:35.978333 32676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:35.978643 32676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:35.979789 32676 hybrid_clock.cc:648] HybridClock initialized: now 1786293275979772 us; error 26 us; skew 500 ppm
Aug 09 16:34:35 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293275, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:35.982069 32676 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:35.982439 32676 webserver.cc:460] Webserver started at http://127.31.15.254:41093/ using document root <none> and password file <none>
I20260809 16:34:35.982635 32676 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:35.982681 32676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:35.982770 32676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:35.983794 32676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "eec251539b834184858771048e9a5f4a"
format_stamp: "Formatted at 2026-08-09 16:34:35 on dist-test-slave-21b9"
server_key: "581dbec41e7e479241b0166355dfac6f"
server_key_iv: "a51398b9d3928700bfd5502291c51abe"
server_key_version: "encryptionkey@0"
I20260809 16:34:35.984081 32676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "eec251539b834184858771048e9a5f4a"
format_stamp: "Formatted at 2026-08-09 16:34:35 on dist-test-slave-21b9"
server_key: "581dbec41e7e479241b0166355dfac6f"
server_key_iv: "a51398b9d3928700bfd5502291c51abe"
server_key_version: "encryptionkey@0"
I20260809 16:34:35.985234 32676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:35.985888 32700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:35.986018 32676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:35.986095 32676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "eec251539b834184858771048e9a5f4a"
format_stamp: "Formatted at 2026-08-09 16:34:35 on dist-test-slave-21b9"
server_key: "581dbec41e7e479241b0166355dfac6f"
server_key_iv: "a51398b9d3928700bfd5502291c51abe"
server_key_version: "encryptionkey@0"
I20260809 16:34:35.986155 32676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:35.994278 32676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:35.995971 32676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:35.996076 32676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:35.999804 32676 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38647
I20260809 16:34:35.999850 321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38647 every 8 connection(s)
I20260809 16:34:36.000123 32676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:36.000954 322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:36.003141 322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a: Bootstrap starting.
I20260809 16:34:36.003717 322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:36.003965 322 log.cc:824] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:36.004124 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32676
I20260809 16:34:36.004200 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:36.004335 31807 external_mini_cluster.cc:1428] Setting key 723794ee34546db86b9a3c497ff58645
I20260809 16:34:36.004659 322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a: No bootstrap required, opened a new log
I20260809 16:34:36.006795 322 raft_consensus.cc:348] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } }
I20260809 16:34:36.006958 322 raft_consensus.cc:374] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:36.006997 322 raft_consensus.cc:729] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec251539b834184858771048e9a5f4a, State: Initialized, Role: FOLLOWER
I20260809 16:34:36.007128 322 consensus_queue.cc:260] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } }
I20260809 16:34:36.007208 322 raft_consensus.cc:388] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:36.007251 322 raft_consensus.cc:482] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:36.007326 322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:36.007884 322 raft_consensus.cc:504] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } }
I20260809 16:34:36.007988 322 leader_election.cc:302] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eec251539b834184858771048e9a5f4a; no voters:
I20260809 16:34:36.008160 322 leader_election.cc:288] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:36.008278 329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:36.008435 322 sys_catalog.cc:564] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:36.008464 329 raft_consensus.cc:686] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [term 1 LEADER]: Becoming Leader. State: Replica: eec251539b834184858771048e9a5f4a, State: Running, Role: LEADER
I20260809 16:34:36.008651 329 consensus_queue.cc:237] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } }
I20260809 16:34:36.009016 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eec251539b834184858771048e9a5f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } } }
I20260809 16:34:36.009188 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:36.009393 329 sys_catalog.cc:455] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eec251539b834184858771048e9a5f4a. Latest consensus state: current_term: 1 leader_uuid: "eec251539b834184858771048e9a5f4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eec251539b834184858771048e9a5f4a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38647 } } }
I20260809 16:34:36.009522 329 sys_catalog.cc:458] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:36.009708 344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:36.010245 344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:36.017616 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.004821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.31.15.254:38647)
0809 16:34:36.004982 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.004991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.005014 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:34:36.005837 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.005843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.005889 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.005996 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.006812 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.008121 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.009205 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.009821 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.010041 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.012765 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.012770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.012770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.012781 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.015601 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.016009 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.016012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.016012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.016117 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.016323 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.016324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.016324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.016627 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.017194 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.017269 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.017409 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260809 16:34:36.018746 344 catalog_manager.cc:1347] Generated new cluster ID: 7662eb3961d645bf8d756fac377e4416
I20260809 16:34:36.018795 344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:36.025885 344 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:36.026355 344 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:36.030301 344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eec251539b834184858771048e9a5f4a: Generated new TSK 0
I20260809 16:34:36.030470 344 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:36.060190 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:36.140393 364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:36.140585 364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:36.140607 364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:36.141750 364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:36.141791 364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:36.141816 364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:36.143293 364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:36.143494 364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:36.143711 364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:36.148307 377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:36.148303 373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:36.148442 364 server_base.cc:1034] running on GCE node
W20260809 16:34:36.148303 374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:36.148712 364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:36.148917 364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:36.150051 364 hybrid_clock.cc:648] HybridClock initialized: now 1786293276150037 us; error 32 us; skew 500 ppm
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:36.152439 364 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:36.152825 364 webserver.cc:460] Webserver started at http://127.31.15.193:41341/ using document root <none> and password file <none>
I20260809 16:34:36.153008 364 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:36.153055 364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:36.153174 364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:36.154125 364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "6e15a4bd7bae4de184819825124ed849"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "858a32da7d5ad0741c6ad7a089d6469c"
server_key_iv: "5bc52388c2a8c417cd023fd2bcdc4a45"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.154553 364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "6e15a4bd7bae4de184819825124ed849"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "858a32da7d5ad0741c6ad7a089d6469c"
server_key_iv: "5bc52388c2a8c417cd023fd2bcdc4a45"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.155916 364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:34:36.156556 388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.156718 364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:36.156798 364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "6e15a4bd7bae4de184819825124ed849"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "858a32da7d5ad0741c6ad7a089d6469c"
server_key_iv: "5bc52388c2a8c417cd023fd2bcdc4a45"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.156857 364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:36.205358 364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:36.208889 364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:36.209055 364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:36.209285 364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:36.209728 364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:36.209769 364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.209798 364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:36.209822 364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.219904 364 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:38945
I20260809 16:34:36.219976 570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:38945 every 8 connection(s)
I20260809 16:34:36.220351 364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:36.231302 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.221145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:45773 (local address 127.31.15.254:38647)
0809 16:34:36.221189 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.221192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.221428 (+ 236us) server_negotiation.cc:407] Connection header received
0809 16:34:36.222393 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.222394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.222407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.222462 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.223329 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.224118 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.225573 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.225876 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.225935 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.228425 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.228427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.228428 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.228436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.229972 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.230359 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.230360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.230361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.230427 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.230634 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.230635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.230635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.230782 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.230856 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.231080 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.231173 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:36.231437 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 364
I20260809 16:34:36.231449 576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.221148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38647 (local address 127.31.15.193:45773)
0809 16:34:36.221305 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.221342 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.222289 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.222562 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.222567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.222914 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:36.223192 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.223199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.224206 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.224207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.225487 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.225490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.225952 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.225953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.226114 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.226658 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:36.226669 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:36.228319 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:36.230064 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.230070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.230072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.230293 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.230487 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.230488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.230488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.230579 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.230841 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:36.230845 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:36.230958 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.231039 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.231182 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 16:34:36.231524 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:36.231678 31807 external_mini_cluster.cc:1428] Setting key afa018f05770fa5e3640fd8aa3fc6cb6
I20260809 16:34:36.232041 571 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38647
I20260809 16:34:36.232138 571 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:36.232321 571 heartbeater.cc:511] Master 127.31.15.254:38647 requested a full tablet report, sending...
I20260809 16:34:36.232847 32732 ts_manager.cc:194] Registered new tserver with Master: 6e15a4bd7bae4de184819825124ed849 (127.31.15.193:38945)
I20260809 16:34:36.233531 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:45773
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:36.260105 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:36.338863 582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:36.339051 582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:36.339083 582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:36.340194 582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:36.340234 582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:36.340268 582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:36.341694 582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:36.341914 582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:36.342115 582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:36.346794 599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:36.346771 592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:36.346838 582 server_base.cc:1034] running on GCE node
W20260809 16:34:36.346776 595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:36.347188 582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:36.347451 582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:36.348590 582 hybrid_clock.cc:648] HybridClock initialized: now 1786293276348578 us; error 29 us; skew 500 ppm
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:36.351006 582 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:36.351456 582 webserver.cc:460] Webserver started at http://127.31.15.194:40595/ using document root <none> and password file <none>
I20260809 16:34:36.351711 582 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:36.351763 582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:36.351887 582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:36.352846 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "1c309fddb9fe44b2a55923b67f716176"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "0bb2f42065496da95f744a8f73a8942d"
server_key_iv: "e242fd869fcee29864d7633a121d8a10"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.353173 582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "1c309fddb9fe44b2a55923b67f716176"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "0bb2f42065496da95f744a8f73a8942d"
server_key_iv: "e242fd869fcee29864d7633a121d8a10"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.354393 582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:36.355068 612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.355223 582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:36.355322 582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "1c309fddb9fe44b2a55923b67f716176"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "0bb2f42065496da95f744a8f73a8942d"
server_key_iv: "e242fd869fcee29864d7633a121d8a10"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.355389 582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:36.371855 582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:36.373801 582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:36.373930 582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:36.374147 582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:36.374447 582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:36.374498 582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.374536 582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:36.374559 582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.379964 582 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:41705
I20260809 16:34:36.380004 807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:41705 every 8 connection(s)
I20260809 16:34:36.380298 582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:36.384670 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 582
I20260809 16:34:36.384749 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:36.384898 31807 external_mini_cluster.cc:1428] Setting key 2198de0a4f634783755e60a55982be07
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:36.389685 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.380955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:58591 (local address 127.31.15.254:38647)
0809 16:34:36.380999 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.381001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.381193 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:34:36.382096 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.382097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.382110 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.382161 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.382722 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.383244 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.384230 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.384417 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.384471 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.386580 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.386581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.386581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.386589 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.387826 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.388289 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.388289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.388290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.388337 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.388715 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.388716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.388716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.388891 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.388965 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.389474 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.389578 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:34:36.390129 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.380917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38647 (local address 127.31.15.194:58591)
0809 16:34:36.381085 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.381118 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.381997 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.382227 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.382231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.382466 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:36.382640 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.382644 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.383358 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.383359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.384153 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.384156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.384482 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.384483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.384618 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.385080 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:36.385087 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:36.386471 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:36.387901 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.387907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.387909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.388223 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.388392 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.388392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.388393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.388615 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.388969 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:36.388983 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:36.389242 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.389456 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.389743 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260809 16:34:36.390959 808 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38647
I20260809 16:34:36.391098 808 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:36.391430 808 heartbeater.cc:511] Master 127.31.15.254:38647 requested a full tablet report, sending...
I20260809 16:34:36.392143 32732 ts_manager.cc:194] Registered new tserver with Master: 1c309fddb9fe44b2a55923b67f716176 (127.31.15.194:41705)
I20260809 16:34:36.392686 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:58591
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:36.418769 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:36.564532 815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:36.564913 815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:36.565008 815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:36.567049 815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:36.567126 815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:36.567209 815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:36.569833 815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:36.570331 815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:36.570799 815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:36.578930 822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:36.578907 823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:36.578907 825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:36.579661 815 server_base.cc:1034] running on GCE node
I20260809 16:34:36.579974 815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:36.580297 815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:36.581485 815 hybrid_clock.cc:648] HybridClock initialized: now 1786293276581458 us; error 43 us; skew 500 ppm
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:36.585975 815 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:36.586697 815 webserver.cc:460] Webserver started at http://127.31.15.195:32871/ using document root <none> and password file <none>
I20260809 16:34:36.587060 815 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:36.587158 815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:36.587409 815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:36.589218 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "086768f514664b96a40fe4c5c103682c"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "46a747525a2b58917c0cee4eab4fad73"
server_key_iv: "d7e03a1071fd6672636d8642319a17df"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.589772 815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "086768f514664b96a40fe4c5c103682c"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "46a747525a2b58917c0cee4eab4fad73"
server_key_iv: "d7e03a1071fd6672636d8642319a17df"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.592201 815 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260809 16:34:36.593451 838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.593703 815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:36.593838 815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "086768f514664b96a40fe4c5c103682c"
format_stamp: "Formatted at 2026-08-09 16:34:36 on dist-test-slave-21b9"
server_key: "46a747525a2b58917c0cee4eab4fad73"
server_key_iv: "d7e03a1071fd6672636d8642319a17df"
server_key_version: "encryptionkey@0"
I20260809 16:34:36.593945 815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:36.619338 815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:36.623050 815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:36.623245 815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:36.623677 815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:36.624166 815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:36.624222 815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.624289 815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:36.624339 815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:36.634289 815 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:41529
I20260809 16:34:36.634389 1035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:41529 every 8 connection(s)
I20260809 16:34:36.634866 815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:36.636838 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 815
I20260809 16:34:36.636999 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:36.637365 31807 external_mini_cluster.cc:1428] Setting key 6c8d6d78700172bb5626c46481658759
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:36.651180 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.636152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:39047 (local address 127.31.15.254:38647)
0809 16:34:36.636232 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.636234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.636615 (+ 381us) server_negotiation.cc:407] Connection header received
0809 16:34:36.638702 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.638703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.638727 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.638818 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.640086 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.640928 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.642651 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.642929 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.643032 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.646617 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.646619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.646629 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.646650 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.648839 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.649810 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.649812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.649812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.649914 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.650291 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.650292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.650292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.650506 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.650613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.650966 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.651081 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:34:36.651571 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.636147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38647 (local address 127.31.15.195:39047)
0809 16:34:36.636383 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.636439 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.638559 (+ 2120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.638951 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.638954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.639640 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:36.639957 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.639962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.641066 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.641067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.642465 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.642467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.643050 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.643051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.643260 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.644060 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:36.644078 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:36.646468 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:36.648979 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.648993 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.648996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.649653 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.649998 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:36.649998 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:36.649999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:36.650173 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:36.650590 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:36.650594 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:36.650786 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.650936 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.651149 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260809 16:34:36.652321 1037 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38647
I20260809 16:34:36.652496 1037 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:36.652824 1037 heartbeater.cc:511] Master 127.31.15.254:38647 requested a full tablet report, sending...
I20260809 16:34:36.653388 32732 ts_manager.cc:194] Registered new tserver with Master: 086768f514664b96a40fe4c5c103682c (127.31.15.195:41529)
I20260809 16:34:36.653878 32732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:39047
I20260809 16:34:36.660953 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:34:36.672919 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.663679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.31.15.254:38647)
0809 16:34:36.663779 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.663781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.663924 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:34:36.664022 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.664023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.664045 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.664116 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.665072 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.665846 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.666786 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.667072 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.667164 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.668977 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.668979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.668979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.668987 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.671127 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.671652 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.671653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.671661 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.671752 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.672040 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.672041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.672041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.672253 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.672391 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.672614 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.672762 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:34:36.676079 32732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:36.678663 32732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:36.689080 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:41529 (local address 127.0.0.1:37566)
0809 16:34:36.682934 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.683005 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.683125 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.683480 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.683483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.683486 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:36.683623 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.683635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685730 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.685731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.686970 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.686979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688438 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.688621 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.688651 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.688871 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.688971 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":40,"thread_start_us":510,"threads_started":1}
I20260809 16:34:36.689399 1054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:41705 (local address 127.0.0.1:42242)
0809 16:34:36.683322 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.683332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.683403 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.683924 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.683925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.683928 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:36.684013 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.684015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685260 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.685261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.687446 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.687449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688984 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.689105 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.689112 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.689208 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.689276 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":66,"thread_start_us":398,"threads_started":1}
I20260809 16:34:36.689483 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.31.15.193:38945)
0809 16:34:36.682652 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.682657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.683105 (+ 448us) server_negotiation.cc:407] Connection header received
0809 16:34:36.683356 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.683358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.683399 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.683482 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:36.683983 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685622 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.687715 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688869 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688977 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.689076 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.689289 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.689353 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":118,"threads_started":1}
I20260809 16:34:36.689656 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.31.15.195:41529)
0809 16:34:36.682884 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.682892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.683209 (+ 317us) server_negotiation.cc:407] Connection header received
0809 16:34:36.683311 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.683314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.683331 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.683479 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:36.683761 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685456 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.687152 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688320 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688438 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.688545 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.689424 (+ 879us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.689511 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":130,"threads_started":1}
I20260809 16:34:36.689868 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:38945 (local address 127.0.0.1:35022)
0809 16:34:36.683015 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.683024 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.683163 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.683751 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.683752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.683755 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:36.683822 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.683824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685778 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.685778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.687616 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.687618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688985 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.689105 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.689114 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.689650 (+ 536us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.689763 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":51,"thread_start_us":124,"threads_started":1}
I20260809 16:34:36.689930 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.682726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42242 (local address 127.31.15.194:41705)
0809 16:34:36.683645 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.683649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.683679 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:34:36.683763 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.683765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.683782 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.683902 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:36.684102 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.685060 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.687561 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.688707 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.688824 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.688929 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.689743 (+ 814us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.689819 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":126,"threads_started":1}
I20260809 16:34:36.690719 915 tablet_service.cc:1467] Processing CreateTablet for tablet 4bead010f79c44e9a610fdc0c4e9cb2a (DEFAULT_TABLE table=test-table [id=7cbef12437174c6c862bfcca7d445655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:36.690909 691 tablet_service.cc:1467] Processing CreateTablet for tablet 4bead010f79c44e9a610fdc0c4e9cb2a (DEFAULT_TABLE table=test-table [id=7cbef12437174c6c862bfcca7d445655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:36.691419 915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bead010f79c44e9a610fdc0c4e9cb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:36.691457 691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bead010f79c44e9a610fdc0c4e9cb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:36.691239 467 tablet_service.cc:1467] Processing CreateTablet for tablet 4bead010f79c44e9a610fdc0c4e9cb2a (DEFAULT_TABLE table=test-table [id=7cbef12437174c6c862bfcca7d445655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:36.691864 467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bead010f79c44e9a610fdc0c4e9cb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:36.696741 1062 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Bootstrap starting.
I20260809 16:34:36.697716 1062 tablet_bootstrap.cc:654] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:36.698236 1063 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Bootstrap starting.
I20260809 16:34:36.698932 1064 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Bootstrap starting.
I20260809 16:34:36.698825 1062 log.cc:824] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:36.700111 1063 tablet_bootstrap.cc:654] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:36.700317 1064 tablet_bootstrap.cc:654] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:36.701021 1062 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: No bootstrap required, opened a new log
I20260809 16:34:36.701164 1062 ts_tablet_manager.cc:1397] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 16:34:36.701329 1064 log.cc:824] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:36.701598 1063 log.cc:824] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:36.702425 1064 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: No bootstrap required, opened a new log
I20260809 16:34:36.702529 1064 ts_tablet_manager.cc:1397] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:34:36.704077 1062 raft_consensus.cc:348] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.704303 1062 raft_consensus.cc:374] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:36.704337 1062 raft_consensus.cc:729] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c309fddb9fe44b2a55923b67f716176, State: Initialized, Role: FOLLOWER
I20260809 16:34:36.704564 1062 consensus_queue.cc:260] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.705055 1062 ts_tablet_manager.cc:1428] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 16:34:36.705111 808 heartbeater.cc:503] Master 127.31.15.254:38647 was elected leader, sending a full tablet report...
I20260809 16:34:36.705838 1063 tablet_bootstrap.cc:492] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: No bootstrap required, opened a new log
I20260809 16:34:36.705920 1063 ts_tablet_manager.cc:1397] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260809 16:34:36.705996 1064 raft_consensus.cc:348] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.706327 1064 raft_consensus.cc:374] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:36.706422 1064 raft_consensus.cc:729] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e15a4bd7bae4de184819825124ed849, State: Initialized, Role: FOLLOWER
I20260809 16:34:36.706647 1064 consensus_queue.cc:260] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.707309 571 heartbeater.cc:503] Master 127.31.15.254:38647 was elected leader, sending a full tablet report...
I20260809 16:34:36.707255 1064 ts_tablet_manager.cc:1428] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 16:34:36.708832 1063 raft_consensus.cc:348] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.709157 1063 raft_consensus.cc:374] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:36.709261 1063 raft_consensus.cc:729] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086768f514664b96a40fe4c5c103682c, State: Initialized, Role: FOLLOWER
I20260809 16:34:36.709437 1063 consensus_queue.cc:260] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.709883 1037 heartbeater.cc:503] Master 127.31.15.254:38647 was elected leader, sending a full tablet report...
I20260809 16:34:36.709856 1063 ts_tablet_manager.cc:1428] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
W20260809 16:34:36.721159 573 tablet.cc:2369] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:34:36.881105 811 tablet.cc:2369] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:34:36.885779 1038 tablet.cc:2369] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:36.941550 1075 raft_consensus.cc:482] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:36.941716 1075 raft_consensus.cc:504] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.942101 1075 leader_election.cc:288] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 086768f514664b96a40fe4c5c103682c (127.31.15.195:41529), 1c309fddb9fe44b2a55923b67f716176 (127.31.15.194:41705)
I20260809 16:34:36.944885 1061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.942383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:55577 (local address 127.31.15.194:41705)
0809 16:34:36.942428 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.942430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.942574 (+ 144us) server_negotiation.cc:407] Connection header received
0809 16:34:36.942623 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.942624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.942635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.942675 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:36.942828 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.943315 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944239 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.944585 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944647 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.944657 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.944786 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.944819 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:36.944901 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.942284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:41705 (local address 127.31.15.193:55577)
0809 16:34:36.942500 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.942509 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.942567 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.942688 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.942689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.942691 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:36.942766 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.942768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.943390 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.943391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.944151 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.944153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.944659 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.944726 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.944730 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.944774 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.944814 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":15,"thread_start_us":107,"threads_started":1}
I20260809 16:34:36.944993 1058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.942378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:54019 (local address 127.31.15.195:41529)
0809 16:34:36.942428 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.942430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.942542 (+ 112us) server_negotiation.cc:407] Connection header received
0809 16:34:36.942606 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.942608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.942617 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.942656 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:36.942834 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.943330 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944312 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.944682 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944745 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.944753 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.944893 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.944929 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:36.945019 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.942284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:41529 (local address 127.31.15.193:54019)
0809 16:34:36.942463 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:36.942474 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:36.942540 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:36.942672 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:36.942673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:36.942675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:36.942770 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.942771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.943406 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.943407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.944231 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:36.944234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.944756 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.944757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:36.944836 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.944839 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:36.944883 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:34:36.944920 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":93,"threads_started":1}
I20260809 16:34:36.945329 957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bead010f79c44e9a610fdc0c4e9cb2a" candidate_uuid: "6e15a4bd7bae4de184819825124ed849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086768f514664b96a40fe4c5c103682c" is_pre_election: true
I20260809 16:34:36.945372 720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bead010f79c44e9a610fdc0c4e9cb2a" candidate_uuid: "6e15a4bd7bae4de184819825124ed849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c309fddb9fe44b2a55923b67f716176" is_pre_election: true
I20260809 16:34:36.945498 957 raft_consensus.cc:2454] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e15a4bd7bae4de184819825124ed849 in term 0.
I20260809 16:34:36.945506 720 raft_consensus.cc:2454] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e15a4bd7bae4de184819825124ed849 in term 0.
I20260809 16:34:36.945753 390 leader_election.cc:302] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c309fddb9fe44b2a55923b67f716176, 6e15a4bd7bae4de184819825124ed849; no voters:
I20260809 16:34:36.945875 1075 raft_consensus.cc:2781] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:36.945936 1075 raft_consensus.cc:482] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:36.945957 1075 raft_consensus.cc:3037] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:36.946636 1075 raft_consensus.cc:504] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.946758 1075 leader_election.cc:288] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [CANDIDATE]: Term 1 election: Requested vote from peers 086768f514664b96a40fe4c5c103682c (127.31.15.195:41529), 1c309fddb9fe44b2a55923b67f716176 (127.31.15.194:41705)
I20260809 16:34:36.946916 720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bead010f79c44e9a610fdc0c4e9cb2a" candidate_uuid: "6e15a4bd7bae4de184819825124ed849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c309fddb9fe44b2a55923b67f716176"
I20260809 16:34:36.946998 720 raft_consensus.cc:3037] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:36.947744 720 raft_consensus.cc:2454] T 4bead010f79c44e9a610fdc0c4e9cb2a P 1c309fddb9fe44b2a55923b67f716176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e15a4bd7bae4de184819825124ed849 in term 1.
I20260809 16:34:36.947850 957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bead010f79c44e9a610fdc0c4e9cb2a" candidate_uuid: "6e15a4bd7bae4de184819825124ed849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "086768f514664b96a40fe4c5c103682c"
I20260809 16:34:36.947916 390 leader_election.cc:302] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c309fddb9fe44b2a55923b67f716176, 6e15a4bd7bae4de184819825124ed849; no voters:
I20260809 16:34:36.947927 957 raft_consensus.cc:3037] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:36.947999 1075 raft_consensus.cc:2781] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:36.948134 1075 raft_consensus.cc:686] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [term 1 LEADER]: Becoming Leader. State: Replica: 6e15a4bd7bae4de184819825124ed849, State: Running, Role: LEADER
I20260809 16:34:36.948287 1075 consensus_queue.cc:237] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } }
I20260809 16:34:36.948714 957 raft_consensus.cc:2454] T 4bead010f79c44e9a610fdc0c4e9cb2a P 086768f514664b96a40fe4c5c103682c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e15a4bd7bae4de184819825124ed849 in term 1.
I20260809 16:34:36.949120 32732 catalog_manager.cc:5495] T 4bead010f79c44e9a610fdc0c4e9cb2a P 6e15a4bd7bae4de184819825124ed849 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e15a4bd7bae4de184819825124ed849 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "6e15a4bd7bae4de184819825124ed849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e15a4bd7bae4de184819825124ed849" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "086768f514664b96a40fe4c5c103682c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c309fddb9fe44b2a55923b67f716176" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41705 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:36.978624 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.971630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.31.15.193:38945)
0809 16:34:36.971691 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.971693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.971810 (+ 117us) server_negotiation.cc:407] Connection header received
0809 16:34:36.971877 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.971878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.971889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.971936 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.972519 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.972977 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.973677 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.973861 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.973914 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.976098 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.976110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.976110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.976118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:36.977588 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.977898 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.977901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.977902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.977957 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:36.978121 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:36.978122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:36.978123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:36.978264 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:36.978427 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:36.978463 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:34:36.978535 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:34:36 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:37.001748 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:36.995387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.31.15.193:38945)
0809 16:34:36.995432 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:36.995434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:36.995529 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:34:36.995590 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:36.995591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:36.995599 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:36.995637 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:36.996140 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.996590 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.997219 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:36.997386 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:36.997427 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:36.999543 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:36.999544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:36.999545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:36.999552 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:37.000753 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:37.001052 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:37.001053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:37.001054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:37.001100 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:37.001256 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:37.001257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:37.001257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:37.001343 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:37.001426 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:37.001588 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:34:37.001663 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:34:37 dist-test-slave-21b9 krb5kdc[32660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:34:37 dist-test-slave-21b9 krb5kdc[32660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:37.022475 1055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:37.016639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.31.15.193:38945)
0809 16:34:37.016682 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:37.016684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:37.016777 (+ 93us) server_negotiation.cc:407] Connection header received
0809 16:34:37.016818 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:37.016819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:37.016827 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:37.016871 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:37.017332 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:37.017766 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:37.018396 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:37.018559 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:37.018598 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:37.020488 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:37.020489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:37.020489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:37.020495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:37.021685 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:37.021934 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:37.021935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:37.021935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:37.021976 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:37.022121 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:37.022122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:37.022122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:37.022215 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:37.022288 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:37.022351 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:34:37.022404 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:34:37.022972 435 tablet_service.cc:2908] Rejecting scan request for tablet 4bead010f79c44e9a610fdc0c4e9cb2a: Uninitialized: clean time has not yet been initialized
I20260809 16:34:37.023802 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 364
I20260809 16:34:37.035007 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 582
I20260809 16:34:37.046128 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 815
I20260809 16:34:37.067800 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32676
2026-08-09T16:34:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2432 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:37 dist-test-slave-21b9 krb5kdc[1114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:37 dist-test-slave-21b9 krb5kdc[1114](info): set up 2 sockets
Aug 09 16:34:37 dist-test-slave-21b9 krb5kdc[1114](info): commencing operation
krb5kdc: starting...
W20260809 16:34:38.218250 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:38.330499 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34589
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:34589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:38.411229 1141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:38.411475 1141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:38.411499 1141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:38.412612 1141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:38.412645 1141 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:34:38.412657 1141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:38.412668 1141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:38.412683 1141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:38.412705 1141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:38.414115 1141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:34589
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:38.414304 1141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:38.414477 1141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:38.419051 1149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.419059 1150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:38.419183 1141 server_base.cc:1034] running on GCE node
W20260809 16:34:38.419059 1152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:38.419471 1141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:38.419777 1141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:38.420922 1141 hybrid_clock.cc:648] HybridClock initialized: now 1786293278420910 us; error 28 us; skew 500 ppm
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:38.427011 1141 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:38.427480 1141 webserver.cc:460] Webserver started at http://127.31.15.254:39075/ using document root <none> and password file <none>
I20260809 16:34:38.427682 1141 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:38.427729 1141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:38.427850 1141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:38.428855 1141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "7237047554664c909687be10b01423c4"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "b07a802755f5bc2d910839a54ad3fd91"
server_key_iv: "6c043d6062ced1907825cf2bcca7706c"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.429200 1141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "7237047554664c909687be10b01423c4"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "b07a802755f5bc2d910839a54ad3fd91"
server_key_iv: "6c043d6062ced1907825cf2bcca7706c"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.430415 1141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:38.431061 1162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.431203 1141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:38.431315 1141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "7237047554664c909687be10b01423c4"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "b07a802755f5bc2d910839a54ad3fd91"
server_key_iv: "6c043d6062ced1907825cf2bcca7706c"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.431378 1141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:38.442767 1141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:38.444691 1141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:38.444826 1141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:38.449371 1141 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:34589
I20260809 16:34:38.449422 1269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:34589 every 8 connection(s)
I20260809 16:34:38.449724 1141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:38.450342 1270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:38.452628 1270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Bootstrap starting.
I20260809 16:34:38.453178 1270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:38.453438 1270 log.cc:824] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:38.454077 1270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: No bootstrap required, opened a new log
I20260809 16:34:38.455080 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1141
I20260809 16:34:38.455179 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:38.455363 31807 external_mini_cluster.cc:1428] Setting key 9a50aa0d7fdf9607bb22138f60f9d7bb
I20260809 16:34:38.456290 1270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } }
I20260809 16:34:38.456426 1270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:38.456447 1270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7237047554664c909687be10b01423c4, State: Initialized, Role: FOLLOWER
I20260809 16:34:38.456574 1270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } }
I20260809 16:34:38.456653 1270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:38.456684 1270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:38.456730 1270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:38.457399 1270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } }
I20260809 16:34:38.457511 1270 leader_election.cc:302] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7237047554664c909687be10b01423c4; no voters:
I20260809 16:34:38.457669 1270 leader_election.cc:288] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:38.457702 1279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:38.457850 1279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [term 1 LEADER]: Becoming Leader. State: Replica: 7237047554664c909687be10b01423c4, State: Running, Role: LEADER
I20260809 16:34:38.457917 1270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:38.457942 1279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } }
I20260809 16:34:38.458293 1279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7237047554664c909687be10b01423c4. Latest consensus state: current_term: 1 leader_uuid: "7237047554664c909687be10b01423c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } } }
I20260809 16:34:38.458366 1279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:38.458513 1285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7237047554664c909687be10b01423c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7237047554664c909687be10b01423c4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34589 } } }
I20260809 16:34:38.458539 1293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:38.458571 1285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:38.459059 1293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:38.460577 1293 catalog_manager.cc:1347] Generated new cluster ID: 4e6ba81bde0f45a4984480028bc0153b
I20260809 16:34:38.460626 1293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:38.468151 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.456986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53340 (local address 127.31.15.254:34589)
0809 16:34:38.457174 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.457183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.457215 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:34:38.457331 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.457338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.457392 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.457517 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.458283 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.459739 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.460640 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.461207 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.461319 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.464004 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.464008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.464009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.464020 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.466591 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.466978 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.466981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.466981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.467076 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.467262 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.467262 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.467262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.467558 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.467710 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.467813 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.467938 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260809 16:34:38.491765 1293 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:38.492302 1293 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:38.502858 1293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Generated new TSK 0
I20260809 16:34:38.503069 1293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:38.530331 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:34589
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:38.611792 1323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:38.612002 1323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:38.612035 1323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:38.613183 1323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:38.613225 1323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:38.613255 1323 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:38.613276 1323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:38.614691 1323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:34589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:38.614918 1323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:38.615134 1323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:38.620112 1333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.620093 1337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.620097 1335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:38.620127 1323 server_base.cc:1034] running on GCE node
I20260809 16:34:38.620534 1323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:38.620762 1323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:38.621912 1323 hybrid_clock.cc:648] HybridClock initialized: now 1786293278621898 us; error 31 us; skew 500 ppm
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:38.624415 1323 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:38.624852 1323 webserver.cc:460] Webserver started at http://127.31.15.193:46251/ using document root <none> and password file <none>
I20260809 16:34:38.625049 1323 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:38.625093 1323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:38.625211 1323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:38.626231 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "0da43d57cf2941fba83928d4c8ac399d"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "06d00cade45b2ba9802844f4b8b4eefd"
server_key_iv: "89ceb65b070d19d9a3c24cfccf5257f9"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.626538 1323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "0da43d57cf2941fba83928d4c8ac399d"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "06d00cade45b2ba9802844f4b8b4eefd"
server_key_iv: "89ceb65b070d19d9a3c24cfccf5257f9"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.627861 1323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:38.628525 1346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.628672 1323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:38.628753 1323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "0da43d57cf2941fba83928d4c8ac399d"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "06d00cade45b2ba9802844f4b8b4eefd"
server_key_iv: "89ceb65b070d19d9a3c24cfccf5257f9"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.628816 1323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:38.638774 1323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:38.640623 1323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:38.640744 1323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:38.641211 1323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:38.641249 1323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.641273 1323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:38.641295 1323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:38.647938 1323 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:36871
I20260809 16:34:38.648201 1537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:36871 every 8 connection(s)
I20260809 16:34:38.648324 1323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:38.650246 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.641705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:43331 (local address 127.31.15.254:34589)
0809 16:34:38.641747 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.641749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.641974 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:34:38.642439 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.642441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.642453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.642546 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.643438 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.644025 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.644979 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.645193 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.645251 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.647759 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.647760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.647761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.647770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.649247 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.649585 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.649585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.649585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.649634 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.649800 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.649801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.649801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.649891 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.649934 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.650124 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.650185 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:38.650409 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.641711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.193:43331)
0809 16:34:38.641864 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.641898 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.642348 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.642655 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.642661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.643016 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.643321 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.643329 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.644111 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.644112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.644897 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.644900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.645277 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.645278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.645404 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.645825 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.645835 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.647657 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.649313 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.649317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.649318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.649527 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.649676 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.649676 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.649676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.649759 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.649932 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.649934 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.650038 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.650111 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.650221 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260809 16:34:38.650849 1538 heartbeater.cc:347] Connected to a master server at 127.31.15.254:34589
I20260809 16:34:38.650929 1538 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:38.651091 1538 heartbeater.cc:511] Master 127.31.15.254:34589 requested a full tablet report, sending...
I20260809 16:34:38.651614 1196 ts_manager.cc:194] Registered new tserver with Master: 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871)
I20260809 16:34:38.652267 1196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:43331
I20260809 16:34:38.656000 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1323
I20260809 16:34:38.656109 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:38.656275 31807 external_mini_cluster.cc:1428] Setting key 2cfa2687ce710183aa026ede929ec4d7
I20260809 16:34:38.657284 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.651870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:38571 (local address 127.31.15.254:34589)
0809 16:34:38.651908 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.651909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.652536 (+ 627us) server_negotiation.cc:407] Connection header received
0809 16:34:38.652593 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.652594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.652604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.652646 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.652989 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.653433 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.654062 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.654229 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.654271 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.654941 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.654942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.654942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.654946 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.656321 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.656625 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.656627 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.656627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.656681 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.656839 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.656840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.656840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.656939 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.656992 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.657127 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.657195 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:34:38.657303 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.651819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.193:38571)
0809 16:34:38.652470 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.652481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.652536 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.652660 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.652661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.652860 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.652923 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.652925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.653509 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.653510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.654007 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.654008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.654274 (+ 266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.654275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.654324 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.654639 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.654642 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.654881 (+ 239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.656394 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.656395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.656395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.656571 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.656735 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.656736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.656736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.656791 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.656999 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.656999 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.657042 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.657099 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.657195 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:38.680033 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:34589
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:38.759104 1551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:38.759339 1551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:38.759373 1551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:38.760537 1551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:38.760576 1551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:38.760604 1551 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:38.760627 1551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:38.762051 1551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:34589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:38.762277 1551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:38.762490 1551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:38.767421 1566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.767474 1557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.767613 1561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:38.767510 1551 server_base.cc:1034] running on GCE node
I20260809 16:34:38.767853 1551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:38.768105 1551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:38.769254 1551 hybrid_clock.cc:648] HybridClock initialized: now 1786293278769237 us; error 30 us; skew 500 ppm
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:38.771679 1551 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:38.772117 1551 webserver.cc:460] Webserver started at http://127.31.15.194:36819/ using document root <none> and password file <none>
I20260809 16:34:38.772349 1551 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:38.772398 1551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:38.772526 1551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:38.773500 1551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "6135d24e35b748d8af686072967663f2"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "8149b031e824dcd57c98cdb4f26c51ec"
server_key_iv: "27f003c74ec42e94ffc1438003eb425f"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.773835 1551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "6135d24e35b748d8af686072967663f2"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "8149b031e824dcd57c98cdb4f26c51ec"
server_key_iv: "27f003c74ec42e94ffc1438003eb425f"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.775063 1551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:38.775794 1578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.775941 1551 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.776026 1551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "6135d24e35b748d8af686072967663f2"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "8149b031e824dcd57c98cdb4f26c51ec"
server_key_iv: "27f003c74ec42e94ffc1438003eb425f"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.776099 1551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:38.790810 1551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:38.792894 1551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:38.793004 1551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:38.793442 1551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:38.793483 1551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.793502 1551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:38.793514 1551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.800136 1551 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:39469
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:38.800215 1777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:39469 every 8 connection(s)
I20260809 16:34:38.800479 1551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:38.803150 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.793901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:46763 (local address 127.31.15.254:34589)
0809 16:34:38.793955 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.793957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.794164 (+ 207us) server_negotiation.cc:407] Connection header received
0809 16:34:38.794653 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.794656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.794669 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.794727 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.795626 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.796390 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.797849 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.798133 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.798202 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.800652 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.800654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.800654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.800663 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.801922 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.802276 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.802278 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.802278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.802336 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.802556 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.802557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.802558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.802668 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.802724 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.802979 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.803060 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:34:38.803319 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.793902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.194:46763)
0809 16:34:38.794043 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.794080 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.794537 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.794843 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.794848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.795193 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.795524 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.795531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.796479 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.796480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.797760 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.797764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.798218 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.798219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.798367 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.798891 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.798902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.800563 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.801996 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.802002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.802004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.802209 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.802390 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.802391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.802391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.802500 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.802722 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.802728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.802856 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.802945 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.803094 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260809 16:34:38.803762 1779 heartbeater.cc:347] Connected to a master server at 127.31.15.254:34589
I20260809 16:34:38.803846 1779 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:38.804021 1779 heartbeater.cc:511] Master 127.31.15.254:34589 requested a full tablet report, sending...
I20260809 16:34:38.804407 1195 ts_manager.cc:194] Registered new tserver with Master: 6135d24e35b748d8af686072967663f2 (127.31.15.194:39469)
I20260809 16:34:38.804484 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1551
I20260809 16:34:38.804581 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:38.804757 31807 external_mini_cluster.cc:1428] Setting key ab639a1bc20ef6ff56b2e79ed8467bc6
I20260809 16:34:38.804888 1195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:46763
I20260809 16:34:38.810173 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.804742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:35175 (local address 127.31.15.254:34589)
0809 16:34:38.805023 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.805025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.805033 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:38.805050 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.805050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.805058 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.805104 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.805485 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.806058 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.806710 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.806948 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.807011 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.807817 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.807819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.807819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.807826 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.809108 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.809470 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.809470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.809471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.809531 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.809697 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.809698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.809698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.809822 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.809866 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.810039 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.810094 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260809 16:34:38.810443 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.804742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.194:35175)
0809 16:34:38.804870 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.804880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.804928 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.805146 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.805147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.805340 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.805397 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.805398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.806158 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.806159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.806645 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.806647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.807016 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.807016 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.807071 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.807487 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.807491 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.807741 (+ 250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.809208 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.809209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.809209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.809407 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.809582 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.809583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.809583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.809647 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.809885 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.809885 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.809947 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.810019 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.810117 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:38.830232 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:34589
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:46243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:38.910845 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:38.911044 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:38.911073 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:38.912231 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:38.912271 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:38.912290 1799 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:38.912307 1799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:38.913713 1799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:34589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:38.913923 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:38.914131 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:38.918888 1809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.918849 1808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:38.918848 1811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:38.919148 1799 server_base.cc:1034] running on GCE node
I20260809 16:34:38.919354 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:38.919571 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:38.920711 1799 hybrid_clock.cc:648] HybridClock initialized: now 1786293278920692 us; error 33 us; skew 500 ppm
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:38.923051 1799 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:38.923503 1799 webserver.cc:460] Webserver started at http://127.31.15.195:38865/ using document root <none> and password file <none>
I20260809 16:34:38.923702 1799 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:38.923749 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:38.923877 1799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:38.924883 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "cb10fb26750a49e2965c083ed191fa8c"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "bc0f06ed4a58ce15f064f3554973bfa2"
server_key_iv: "07fd638c904040be4dc5890cdaa698bf"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.925310 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "cb10fb26750a49e2965c083ed191fa8c"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "bc0f06ed4a58ce15f064f3554973bfa2"
server_key_iv: "07fd638c904040be4dc5890cdaa698bf"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.926509 1799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:38.927137 1821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.927315 1799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:38.927400 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "cb10fb26750a49e2965c083ed191fa8c"
format_stamp: "Formatted at 2026-08-09 16:34:38 on dist-test-slave-21b9"
server_key: "bc0f06ed4a58ce15f064f3554973bfa2"
server_key_iv: "07fd638c904040be4dc5890cdaa698bf"
server_key_version: "encryptionkey@0"
I20260809 16:34:38.927461 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:38.937690 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:38.939319 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:38.939450 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:38.939906 1799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:38.939949 1799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:38.939978 1799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:38.940011 1799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:38.946695 1799 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:38147
I20260809 16:34:38.946755 2033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:38147 every 8 connection(s)
I20260809 16:34:38.947093 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:38.949270 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.940342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:47741 (local address 127.31.15.254:34589)
0809 16:34:38.940430 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.940432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.940633 (+ 201us) server_negotiation.cc:407] Connection header received
0809 16:34:38.941261 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.941263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.941275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.941334 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.942156 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.942906 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.943974 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.944178 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.944234 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.946663 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.946664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.946665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.946672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.948053 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.948404 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.948405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.948406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.948464 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.948673 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.948674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.948674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.948786 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.948841 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.949091 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.949185 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:34:38.949425 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.940313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.195:47741)
0809 16:34:38.940489 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.940545 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.941144 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.941427 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.941431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.941782 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.942054 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.942060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.942986 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.942988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.943889 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.943892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.944261 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.944262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.944368 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.944850 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.944860 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.946571 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.948127 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.948133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.948136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.948337 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.948517 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.948518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.948518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.948618 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.948839 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.948845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.948974 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.949060 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.949215 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260809 16:34:38.949869 2034 heartbeater.cc:347] Connected to a master server at 127.31.15.254:34589
I20260809 16:34:38.949946 2034 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:38.950109 2034 heartbeater.cc:511] Master 127.31.15.254:34589 requested a full tablet report, sending...
I20260809 16:34:38.950489 1196 ts_manager.cc:194] Registered new tserver with Master: cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147)
I20260809 16:34:38.950999 1196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:47741
I20260809 16:34:38.955056 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1799
I20260809 16:34:38.955153 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:38.955365 31807 external_mini_cluster.cc:1428] Setting key 96252cc76072e43fda4ed97f63599588
I20260809 16:34:38.955875 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.950847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:60993 (local address 127.31.15.254:34589)
0809 16:34:38.950891 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.950893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.950955 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:34:38.951009 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.951009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.951016 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.951060 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.951400 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.952046 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.952700 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.952872 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.952917 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.953591 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.953592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.953593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.953598 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.954765 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.955111 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.955112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.955112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.955169 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.955380 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.955381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.955382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.955520 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.955585 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.955722 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.955794 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:34:38.955878 2044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.950754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.31.15.195:60993)
0809 16:34:38.950901 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.950911 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.950967 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.951063 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.951064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.951258 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:38.951344 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.951346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.952116 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.952117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.952634 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.952636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.952920 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.952921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.952971 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.953266 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:38.953269 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:38.953526 (+ 257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:38.954832 (+ 1306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.954833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.954833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.955047 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.955243 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:38.955244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:38.955244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:38.955326 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:38.955588 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:38.955589 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:38.955636 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.955685 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.955781 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260809 16:34:38.956218 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:34:38 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:38.978519 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.972467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.31.15.254:34589)
0809 16:34:38.972521 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.972523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.972533 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:38.972591 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.972592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.972601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.972639 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:38.973106 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.973600 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.974243 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.974427 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.974468 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.976487 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:38.976489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:38.976489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:38.976495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:38.977540 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.977810 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.977810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.977810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.977853 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:38.978084 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:38.978086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:38.978086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:38.978206 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:38.978298 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.978369 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.978453 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:34:38.980209 1196 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:38.981753 1196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:38.987599 2065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.983693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38147 (local address 127.0.0.1:45370)
0809 16:34:38.984060 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.984071 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.984124 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.984309 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.984312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.984315 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:38.984383 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.984385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985333 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.985334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.986387 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.986390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987257 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.987387 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987407 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.987467 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.987518 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":3,"thread_start_us":58,"threads_started":1}
I20260809 16:34:38.987612 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.983862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.31.15.195:38147)
0809 16:34:38.984109 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.984114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.984125 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:38.984201 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.984204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.984218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.984292 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:38.984490 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985259 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.986479 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987186 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987255 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987346 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.987489 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.987526 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":104,"threads_started":1}
I20260809 16:34:38.987767 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.983896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36871 (local address 127.0.0.1:42350)
0809 16:34:38.984199 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.984207 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.984239 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.984549 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.984550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.984552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:38.984594 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.984596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985886 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.985887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.986708 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.986710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987556 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.987624 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987628 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.987658 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.987696 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":32,"threads_started":1}
I20260809 16:34:38.988080 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.984062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.31.15.193:36871)
0809 16:34:38.984350 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.984354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.984364 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:38.984398 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.984400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.984414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.984490 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:38.984674 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985770 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.986781 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987447 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987841 (+ 394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987915 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.987941 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.987982 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":91,"threads_started":1}
I20260809 16:34:38.988147 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.983693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:39469 (local address 127.0.0.1:40502)
0809 16:34:38.983990 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:38.984029 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:38.984101 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:38.984359 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:38.984360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:38.984362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:38.984429 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.984431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985855 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.985856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.986938 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:38.986941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987913 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:38.987976 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987981 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:34:38.988037 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:34:38.988075 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":31,"thread_start_us":56,"threads_started":1}
I20260809 16:34:38.988188 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:38.983782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.31.15.194:39469)
0809 16:34:38.983966 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:38.983971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:38.984180 (+ 209us) server_negotiation.cc:407] Connection header received
0809 16:34:38.984218 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:38.984220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:38.984234 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:38.984298 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:38.984644 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.985770 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987029 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:38.987829 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:38.987894 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:38.987939 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:38.988059 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 16:34:38.988094 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260809 16:34:38.988780 1668 tablet_service.cc:1467] Processing CreateTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (DEFAULT_TABLE table=test-table [id=5709a1966b0c45e99f187367f2539b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:38.988752 1428 tablet_service.cc:1467] Processing CreateTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (DEFAULT_TABLE table=test-table [id=5709a1966b0c45e99f187367f2539b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:38.989006 1910 tablet_service.cc:1467] Processing CreateTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (DEFAULT_TABLE table=test-table [id=5709a1966b0c45e99f187367f2539b8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:38.989144 1668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:38.989144 1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:38.989326 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:38.992033 2075 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Bootstrap starting.
I20260809 16:34:38.992729 2076 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Bootstrap starting.
I20260809 16:34:38.992939 2079 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Bootstrap starting.
I20260809 16:34:38.993105 2075 tablet_bootstrap.cc:654] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:38.993402 2076 tablet_bootstrap.cc:654] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:38.993472 2079 tablet_bootstrap.cc:654] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:38.993547 2075 log.cc:824] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:38.993760 2079 log.cc:824] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:38.993844 2076 log.cc:824] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:38.994416 2075 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: No bootstrap required, opened a new log
I20260809 16:34:38.994416 2079 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: No bootstrap required, opened a new log
I20260809 16:34:38.994496 2075 ts_tablet_manager.cc:1397] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:34:38.994493 2079 ts_tablet_manager.cc:1397] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:38.995772 2076 tablet_bootstrap.cc:492] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: No bootstrap required, opened a new log
I20260809 16:34:38.995834 2076 ts_tablet_manager.cc:1397] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:34:38.996425 2079 raft_consensus.cc:348] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.996425 2075 raft_consensus.cc:348] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.996599 2075 raft_consensus.cc:374] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:38.996599 2079 raft_consensus.cc:374] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:38.996636 2075 raft_consensus.cc:729] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb10fb26750a49e2965c083ed191fa8c, State: Initialized, Role: FOLLOWER
I20260809 16:34:38.996636 2079 raft_consensus.cc:729] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6135d24e35b748d8af686072967663f2, State: Initialized, Role: FOLLOWER
I20260809 16:34:38.996752 2079 consensus_queue.cc:260] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.996752 2075 consensus_queue.cc:260] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.997033 2079 ts_tablet_manager.cc:1428] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:34:38.997139 2075 ts_tablet_manager.cc:1428] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:34:38.997215 2034 heartbeater.cc:503] Master 127.31.15.254:34589 was elected leader, sending a full tablet report...
I20260809 16:34:38.997257 1779 heartbeater.cc:503] Master 127.31.15.254:34589 was elected leader, sending a full tablet report...
I20260809 16:34:38.997423 2076 raft_consensus.cc:348] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.997593 2076 raft_consensus.cc:374] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:38.997633 2076 raft_consensus.cc:729] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da43d57cf2941fba83928d4c8ac399d, State: Initialized, Role: FOLLOWER
I20260809 16:34:38.997745 2076 consensus_queue.cc:260] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:38.997993 2076 ts_tablet_manager.cc:1428] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:34:38.998042 1538 heartbeater.cc:503] Master 127.31.15.254:34589 was elected leader, sending a full tablet report...
I20260809 16:34:39.010268 2084 raft_consensus.cc:482] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:39.010390 2084 raft_consensus.cc:504] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:39.010687 2084 leader_election.cc:288] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147), 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871)
I20260809 16:34:39.014468 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.010939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:53601 (local address 127.31.15.193:36871)
0809 16:34:39.010979 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.010980 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.011216 (+ 236us) server_negotiation.cc:407] Connection header received
0809 16:34:39.011252 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.011253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.011261 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.011332 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:39.011425 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.012147 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.013555 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.014106 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.014170 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.014177 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.014374 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.014419 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:34:39.014499 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.010852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38147 (local address 127.31.15.194:50489)
0809 16:34:39.010898 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.010911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.010951 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.011127 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.011128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.011130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:39.011213 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.011215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.012225 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.012226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.013439 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.013442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.014197 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.014198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.014303 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.014307 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.014368 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.014420 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":8}
I20260809 16:34:39.014652 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.010967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:50489 (local address 127.31.15.195:38147)
0809 16:34:39.011008 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.011010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.011018 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:39.011044 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.011045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.011054 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.011108 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:39.011305 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.012139 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.013539 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.014094 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.014169 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.014177 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.014550 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.014592 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:39.014730 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.010962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36871 (local address 127.31.15.194:53601)
0809 16:34:39.011148 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.011157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.011211 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.011332 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.011333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.011334 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:39.011384 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.011384 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.012225 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.012226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.013439 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.013442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.014238 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.014238 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.014304 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.014308 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.014368 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.014444 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":72,"thread_start_us":127,"threads_started":1}
I20260809 16:34:39.014909 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b67ea9fcc014681ac2d1eaf134ea6ca" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb10fb26750a49e2965c083ed191fa8c" is_pre_election: true
I20260809 16:34:39.015038 1949 raft_consensus.cc:2454] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 0.
I20260809 16:34:39.015084 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b67ea9fcc014681ac2d1eaf134ea6ca" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d" is_pre_election: true
I20260809 16:34:39.015214 1459 raft_consensus.cc:2454] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 0.
I20260809 16:34:39.015318 1582 leader_election.cc:302] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6135d24e35b748d8af686072967663f2, cb10fb26750a49e2965c083ed191fa8c; no voters:
I20260809 16:34:39.015481 2084 raft_consensus.cc:2781] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:39.015535 2084 raft_consensus.cc:482] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:39.015558 2084 raft_consensus.cc:3037] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.016431 2084 raft_consensus.cc:504] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:39.016553 2084 leader_election.cc:288] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 election: Requested vote from peers cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147), 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871)
I20260809 16:34:39.016691 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b67ea9fcc014681ac2d1eaf134ea6ca" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d"
I20260809 16:34:39.016705 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b67ea9fcc014681ac2d1eaf134ea6ca" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb10fb26750a49e2965c083ed191fa8c"
I20260809 16:34:39.016767 1459 raft_consensus.cc:3037] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.016769 1949 raft_consensus.cc:3037] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.017437 1949 raft_consensus.cc:2454] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 1.
I20260809 16:34:39.017468 1459 raft_consensus.cc:2454] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 1.
I20260809 16:34:39.017596 1582 leader_election.cc:302] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6135d24e35b748d8af686072967663f2, cb10fb26750a49e2965c083ed191fa8c; no voters:
I20260809 16:34:39.017673 2084 raft_consensus.cc:2781] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:39.017827 2084 raft_consensus.cc:686] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6135d24e35b748d8af686072967663f2, State: Running, Role: LEADER
I20260809 16:34:39.017911 2084 consensus_queue.cc:237] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } }
I20260809 16:34:39.018668 1196 catalog_manager.cc:5495] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6135d24e35b748d8af686072967663f2 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } health_report { overall_health: UNKNOWN } } }
I20260809 16:34:39.027333 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.025000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.31.15.194:39469)
0809 16:34:39.025078 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.025079 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.025088 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:39.025171 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.025172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.025188 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.025241 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:39.025367 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.025835 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.026583 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.026763 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.026816 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.027005 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.027095 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.027163 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.027207 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.027229 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:34:39.028936 1949 raft_consensus.cc:1261] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Refusing update from remote peer 6135d24e35b748d8af686072967663f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:39.028936 1459 raft_consensus.cc:1261] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Refusing update from remote peer 6135d24e35b748d8af686072967663f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:39.029224 2084 consensus_queue.cc:1035] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.029345 2091 consensus_queue.cc:1035] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.032253 2097 mvcc.cc:205] Tried to move back new op lower bound from 7316657270900178944 to 7316657270857998336. Current Snapshot: MvccSnapshot[applied={T|T < 7316657270900178944}]
I20260809 16:34:39.035283 1196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:39.035391 1196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:39.036134 1196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Sending DeleteTablet for 3 replicas of tablet 7b67ea9fcc014681ac2d1eaf134ea6ca
I20260809 16:34:39.036484 1668 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:40502
I20260809 16:34:39.036485 1910 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45370
I20260809 16:34:39.036737 2111 tablet_replica.cc:330] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: stopping tablet replica
I20260809 16:34:39.036748 1428 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b67ea9fcc014681ac2d1eaf134ea6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:42350
I20260809 16:34:39.036913 2111 raft_consensus.cc:2229] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:39.036906 1196 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:39.037070 1196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:39.037235 2111 raft_consensus.cc:2258] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.037269 2112 tablet_replica.cc:330] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: stopping tablet replica
I20260809 16:34:39.037358 2112 raft_consensus.cc:2229] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:39.037464 2112 raft_consensus.cc:2258] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.037858 2111 ts_tablet_manager.cc:1905] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.037966 2112 ts_tablet_manager.cc:1905] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.038767 1668 tablet_service.cc:1467] Processing CreateTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc (DEFAULT_TABLE table=test-table [id=6b22760e9f27439088253f56cfdf66e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:39.038906 1668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f6ca0c3693430bb3181e092da3d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.039917 2111 ts_tablet_manager.cc:1918] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:39.039917 2112 ts_tablet_manager.cc:1918] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:39.039997 2112 log.cc:1196] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/7b67ea9fcc014681ac2d1eaf134ea6ca
I20260809 16:34:39.040096 2111 log.cc:1196] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/7b67ea9fcc014681ac2d1eaf134ea6ca
I20260809 16:34:39.040254 2112 ts_tablet_manager.cc:1939] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 0da43d57cf2941fba83928d4c8ac399d: Deleting consensus metadata
I20260809 16:34:39.040263 2079 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Bootstrap starting.
I20260809 16:34:39.040438 2111 ts_tablet_manager.cc:1939] T 7b67ea9fcc014681ac2d1eaf134ea6ca P cb10fb26750a49e2965c083ed191fa8c: Deleting consensus metadata
I20260809 16:34:39.040665 2079 tablet_bootstrap.cc:654] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.040833 1166 catalog_manager.cc:4841] TS cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147): tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (table test-table [id=5709a1966b0c45e99f187367f2539b8f]) successfully deleted
I20260809 16:34:39.041265 1428 tablet_service.cc:1467] Processing CreateTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc (DEFAULT_TABLE table=test-table [id=6b22760e9f27439088253f56cfdf66e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:39.041317 2079 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: No bootstrap required, opened a new log
I20260809 16:34:39.041361 2079 ts_tablet_manager.cc:1397] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:39.041369 1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f6ca0c3693430bb3181e092da3d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.041497 2079 raft_consensus.cc:348] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.041541 2079 raft_consensus.cc:374] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.041553 2079 raft_consensus.cc:729] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6135d24e35b748d8af686072967663f2, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.041589 2079 consensus_queue.cc:260] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.041667 2079 ts_tablet_manager.cc:1428] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:39.042481 2076 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Bootstrap starting.
I20260809 16:34:39.042536 1910 tablet_service.cc:1467] Processing CreateTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc (DEFAULT_TABLE table=test-table [id=6b22760e9f27439088253f56cfdf66e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:39.042644 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f6ca0c3693430bb3181e092da3d1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.042847 2076 tablet_bootstrap.cc:654] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.043432 2076 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: No bootstrap required, opened a new log
I20260809 16:34:39.043426 1164 catalog_manager.cc:4841] TS 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871): tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (table test-table [id=5709a1966b0c45e99f187367f2539b8f]) successfully deleted
I20260809 16:34:39.043475 2076 ts_tablet_manager.cc:1397] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:39.043610 2076 raft_consensus.cc:348] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.043674 2076 raft_consensus.cc:374] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.043694 2076 raft_consensus.cc:729] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da43d57cf2941fba83928d4c8ac399d, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.043730 2076 consensus_queue.cc:260] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.043782 2075 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Bootstrap starting.
I20260809 16:34:39.043802 2076 ts_tablet_manager.cc:1428] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:39.044166 2075 tablet_bootstrap.cc:654] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.044508 2110 tablet_replica.cc:330] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: stopping tablet replica
I20260809 16:34:39.044588 2110 raft_consensus.cc:2229] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:39.044697 2110 raft_consensus.cc:2258] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.044775 2075 tablet_bootstrap.cc:492] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: No bootstrap required, opened a new log
I20260809 16:34:39.044816 2075 ts_tablet_manager.cc:1397] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:39.044960 2075 raft_consensus.cc:348] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.045028 2075 raft_consensus.cc:374] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.045047 2075 raft_consensus.cc:729] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb10fb26750a49e2965c083ed191fa8c, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.045096 2075 consensus_queue.cc:260] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.045097 2110 ts_tablet_manager.cc:1905] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.045182 2075 ts_tablet_manager.cc:1428] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:39.046425 2110 ts_tablet_manager.cc:1918] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:39.046485 2110 log.cc:1196] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/7b67ea9fcc014681ac2d1eaf134ea6ca
I20260809 16:34:39.046686 2110 ts_tablet_manager.cc:1939] T 7b67ea9fcc014681ac2d1eaf134ea6ca P 6135d24e35b748d8af686072967663f2: Deleting consensus metadata
I20260809 16:34:39.047073 1164 catalog_manager.cc:4841] TS 6135d24e35b748d8af686072967663f2 (127.31.15.194:39469): tablet 7b67ea9fcc014681ac2d1eaf134ea6ca (table test-table [id=5709a1966b0c45e99f187367f2539b8f]) successfully deleted
W20260809 16:34:39.050948 1782 tablet.cc:2369] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:39.132506 2086 raft_consensus.cc:482] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:39.132618 2086 raft_consensus.cc:504] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.132943 2086 leader_election.cc:288] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871), 6135d24e35b748d8af686072967663f2 (127.31.15.194:39469)
I20260809 16:34:39.136453 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.133480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:46965 (local address 127.31.15.194:39469)
0809 16:34:39.133524 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.133526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.133533 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:39.133559 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.133560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.133569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.133617 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:39.133735 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.134403 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.135523 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.136050 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.136122 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.136131 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.136335 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.136376 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:34:39.136533 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.133124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36871 (local address 127.31.15.195:40441)
0809 16:34:39.133179 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.133191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.133233 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.133424 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.133425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.133427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:39.133499 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.133500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.134406 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.134407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.135587 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.135590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.136267 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.136268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.136352 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.136355 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.136400 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.136437 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:34:39.136459 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.133247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:39469 (local address 127.31.15.195:46965)
0809 16:34:39.133356 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.133365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.133406 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.133620 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.133620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.133622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:39.133686 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.133687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.134515 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.134516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.135444 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.135446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.136150 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.136151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.136223 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.136228 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.136316 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.136356 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260809 16:34:39.136590 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.133210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:40441 (local address 127.31.15.193:36871)
0809 16:34:39.133255 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.133257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.133286 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:34:39.133313 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.133314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.133325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.133371 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:39.133562 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.134321 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.135672 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.136199 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.136258 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.136266 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.136490 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.136530 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:34:39.136811 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f6ca0c3693430bb3181e092da3d1bc" candidate_uuid: "cb10fb26750a49e2965c083ed191fa8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d" is_pre_election: true
I20260809 16:34:39.136817 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f6ca0c3693430bb3181e092da3d1bc" candidate_uuid: "cb10fb26750a49e2965c083ed191fa8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6135d24e35b748d8af686072967663f2" is_pre_election: true
I20260809 16:34:39.136916 1459 raft_consensus.cc:2454] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb10fb26750a49e2965c083ed191fa8c in term 0.
I20260809 16:34:39.136937 1692 raft_consensus.cc:2454] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb10fb26750a49e2965c083ed191fa8c in term 0.
I20260809 16:34:39.137113 1822 leader_election.cc:302] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6135d24e35b748d8af686072967663f2, cb10fb26750a49e2965c083ed191fa8c; no voters:
I20260809 16:34:39.137259 2086 raft_consensus.cc:2781] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:39.137312 2086 raft_consensus.cc:482] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:39.137331 2086 raft_consensus.cc:3037] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.138012 2086 raft_consensus.cc:504] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.138120 2086 leader_election.cc:288] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [CANDIDATE]: Term 1 election: Requested vote from peers 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871), 6135d24e35b748d8af686072967663f2 (127.31.15.194:39469)
I20260809 16:34:39.138289 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f6ca0c3693430bb3181e092da3d1bc" candidate_uuid: "cb10fb26750a49e2965c083ed191fa8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d"
I20260809 16:34:39.138296 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f6ca0c3693430bb3181e092da3d1bc" candidate_uuid: "cb10fb26750a49e2965c083ed191fa8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6135d24e35b748d8af686072967663f2"
I20260809 16:34:39.138366 1459 raft_consensus.cc:3037] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.138366 1692 raft_consensus.cc:3037] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.138957 1692 raft_consensus.cc:2454] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb10fb26750a49e2965c083ed191fa8c in term 1.
I20260809 16:34:39.139119 1822 leader_election.cc:302] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6135d24e35b748d8af686072967663f2, cb10fb26750a49e2965c083ed191fa8c; no voters:
I20260809 16:34:39.139185 1459 raft_consensus.cc:2454] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb10fb26750a49e2965c083ed191fa8c in term 1.
I20260809 16:34:39.139195 2086 raft_consensus.cc:2781] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:39.139384 2086 raft_consensus.cc:686] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 1 LEADER]: Becoming Leader. State: Replica: cb10fb26750a49e2965c083ed191fa8c, State: Running, Role: LEADER
I20260809 16:34:39.139459 2086 consensus_queue.cc:237] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } }
I20260809 16:34:39.140203 1196 catalog_manager.cc:5495] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c reported cstate change: term changed from 0 to 1, leader changed from <none> to cb10fb26750a49e2965c083ed191fa8c (127.31.15.195). New cstate: current_term: 1 leader_uuid: "cb10fb26750a49e2965c083ed191fa8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } health_report { overall_health: HEALTHY } } }
W20260809 16:34:39.149119 1540 tablet.cc:2369] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:34:39 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:39.187713 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.179980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34589 (local address 127.0.0.1:53352)
0809 16:34:39.180149 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.180158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.180226 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.180348 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.180349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.180689 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:39.180740 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.180743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.181406 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.181407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.182126 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.182130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.182633 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.182634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.182685 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.183199 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:39.183204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:39.184684 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:39.186507 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:39.186515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:39.186518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:39.186769 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:39.186971 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:39.186972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:39.186972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:39.187048 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:39.187353 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:39.187361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:39.187408 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.187471 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.187586 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260809 16:34:39.187713 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.180048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.31.15.254:34589)
0809 16:34:39.180115 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.180118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.180216 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:34:39.180289 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.180290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.180299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.180338 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:39.180820 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.181311 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.182215 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.182496 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.182567 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.184768 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:39.184770 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:39.184770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:39.184777 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:39.186443 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.186845 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.186846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.186847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.186922 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.187109 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.187110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.187110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.187248 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:39.187334 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.187512 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.187640 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":35}
I20260809 16:34:39.192112 1196 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53352:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:34:39.192358 1196 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:39.193799 1428 tablet_service.cc:1467] Processing CreateTablet for tablet 3951a4955cd843d69c37ace50586e02e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76eaa67cc06f4e759ef85e1f0fc4a9eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:39.193830 1910 tablet_service.cc:1467] Processing CreateTablet for tablet 3951a4955cd843d69c37ace50586e02e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76eaa67cc06f4e759ef85e1f0fc4a9eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:39.193969 1910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3951a4955cd843d69c37ace50586e02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.193969 1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3951a4955cd843d69c37ace50586e02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.195032 2075 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: Bootstrap starting.
I20260809 16:34:39.195040 2076 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: Bootstrap starting.
I20260809 16:34:39.195127 1668 tablet_service.cc:1467] Processing CreateTablet for tablet 3951a4955cd843d69c37ace50586e02e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76eaa67cc06f4e759ef85e1f0fc4a9eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:39.195261 1668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3951a4955cd843d69c37ace50586e02e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:39.195466 2075 tablet_bootstrap.cc:654] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.195483 2076 tablet_bootstrap.cc:654] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.196141 2076 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: No bootstrap required, opened a new log
I20260809 16:34:39.196184 2076 ts_tablet_manager.cc:1397] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:39.196329 2076 raft_consensus.cc:348] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.196386 2076 raft_consensus.cc:374] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.196403 2076 raft_consensus.cc:729] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da43d57cf2941fba83928d4c8ac399d, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.196447 2076 consensus_queue.cc:260] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.196535 2076 ts_tablet_manager.cc:1428] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:39.196633 2079 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: Bootstrap starting.
I20260809 16:34:39.196959 2079 tablet_bootstrap.cc:654] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:39.197276 2088 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.197369 2088 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.197492 2079 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: No bootstrap required, opened a new log
I20260809 16:34:39.197559 2079 ts_tablet_manager.cc:1397] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
W20260809 16:34:39.197629 2036 tablet.cc:2369] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:39.197666 2079 raft_consensus.cc:348] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.197722 2079 raft_consensus.cc:374] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.197739 2079 raft_consensus.cc:729] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6135d24e35b748d8af686072967663f2, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.197789 2075 tablet_bootstrap.cc:492] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: No bootstrap required, opened a new log
I20260809 16:34:39.197783 2079 consensus_queue.cc:260] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.197834 2075 ts_tablet_manager.cc:1397] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 16:34:39.197870 2079 ts_tablet_manager.cc:1428] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:39.197908 2107 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.198006 2107 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.197985 2075 raft_consensus.cc:348] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.198045 2075 raft_consensus.cc:374] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:39.198074 2075 raft_consensus.cc:729] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb10fb26750a49e2965c083ed191fa8c, State: Initialized, Role: FOLLOWER
I20260809 16:34:39.198117 2075 consensus_queue.cc:260] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.198211 2075 ts_tablet_manager.cc:1428] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:39.198237 2086 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.198304 2086 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.543895 2107 raft_consensus.cc:482] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:39.544008 2107 raft_consensus.cc:504] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.544193 2107 leader_election.cc:288] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871), cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147)
I20260809 16:34:39.544440 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3951a4955cd843d69c37ace50586e02e" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d" is_pre_election: true
I20260809 16:34:39.544443 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3951a4955cd843d69c37ace50586e02e" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb10fb26750a49e2965c083ed191fa8c" is_pre_election: true
I20260809 16:34:39.544551 1949 raft_consensus.cc:2454] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 0.
I20260809 16:34:39.544551 1459 raft_consensus.cc:2454] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 0.
I20260809 16:34:39.544724 1582 leader_election.cc:302] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6135d24e35b748d8af686072967663f2, cb10fb26750a49e2965c083ed191fa8c; no voters:
I20260809 16:34:39.544828 2107 raft_consensus.cc:2781] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:39.544885 2107 raft_consensus.cc:482] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:39.544911 2107 raft_consensus.cc:3037] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.545646 2107 raft_consensus.cc:504] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.545764 2107 leader_election.cc:288] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 election: Requested vote from peers 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871), cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147)
I20260809 16:34:39.545912 1459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3951a4955cd843d69c37ace50586e02e" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da43d57cf2941fba83928d4c8ac399d"
I20260809 16:34:39.545943 1949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3951a4955cd843d69c37ace50586e02e" candidate_uuid: "6135d24e35b748d8af686072967663f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb10fb26750a49e2965c083ed191fa8c"
I20260809 16:34:39.545982 1459 raft_consensus.cc:3037] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.546003 1949 raft_consensus.cc:3037] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:39.546563 1459 raft_consensus.cc:2454] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 1.
I20260809 16:34:39.546643 1949 raft_consensus.cc:2454] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6135d24e35b748d8af686072967663f2 in term 1.
I20260809 16:34:39.546736 1580 leader_election.cc:302] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da43d57cf2941fba83928d4c8ac399d, 6135d24e35b748d8af686072967663f2; no voters:
I20260809 16:34:39.546844 2107 raft_consensus.cc:2781] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:39.546941 2107 raft_consensus.cc:686] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6135d24e35b748d8af686072967663f2, State: Running, Role: LEADER
I20260809 16:34:39.546989 2107 consensus_queue.cc:237] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } }
I20260809 16:34:39.547298 2139 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: TxnStatusTablet state changed. Reason: New leader 6135d24e35b748d8af686072967663f2. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.547381 2139 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:39.547565 2144 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:34:39.547691 2144 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:34:39.547621 1196 catalog_manager.cc:5495] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6135d24e35b748d8af686072967663f2 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } health_report { overall_health: HEALTHY } } }
I20260809 16:34:39.600883 2086 consensus_queue.cc:1035] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.602911 2119 consensus_queue.cc:1035] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.637398 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.634317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:39469 (local address 127.0.0.1:40530)
0809 16:34:39.634417 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.634429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.634492 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.634665 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.634666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.634667 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:39.634744 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.634747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.635482 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.635484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.636399 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.636402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.636960 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.636961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.637021 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.637039 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.637228 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.637236 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.637277 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.637331 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 16:34:39.637383 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.634377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40530 (local address 127.31.15.194:39469)
0809 16:34:39.634423 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.634425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.634468 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:34:39.634557 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.634558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.634569 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.634625 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:39.634809 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.635317 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.636505 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.636751 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.636824 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.637123 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.637172 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.637218 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.637280 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.637303 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:39.638499 1949 raft_consensus.cc:1261] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Refusing update from remote peer 6135d24e35b748d8af686072967663f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:39.638530 1459 raft_consensus.cc:1261] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Refusing update from remote peer 6135d24e35b748d8af686072967663f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:39.638749 2107 consensus_queue.cc:1035] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.638880 2139 consensus_queue.cc:1035] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:39.639310 2088 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: TxnStatusTablet state changed. Reason: New leader 6135d24e35b748d8af686072967663f2. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.639410 2088 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.639555 2086 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: TxnStatusTablet state changed. Reason: New leader 6135d24e35b748d8af686072967663f2. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.639653 2086 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.640368 2139 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.640453 2139 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:39.640502 2139 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.640547 2139 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 6135d24e35b748d8af686072967663f2: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:39.640623 2150 mvcc.cc:205] Tried to move back new op lower bound from 7316657273397751808 to 7316657273024950272. Current Snapshot: MvccSnapshot[applied={T|T < 7316657273397751808}]
I20260809 16:34:39.640729 2086 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.640798 2086 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P cb10fb26750a49e2965c083ed191fa8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.641481 2088 tablet_replica.cc:439] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6135d24e35b748d8af686072967663f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0da43d57cf2941fba83928d4c8ac399d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36871 } } peers { permanent_uuid: "cb10fb26750a49e2965c083ed191fa8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38147 } } peers { permanent_uuid: "6135d24e35b748d8af686072967663f2" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 39469 } } }
I20260809 16:34:39.641571 2088 tablet_replica.cc:442] T 3951a4955cd843d69c37ace50586e02e P 0da43d57cf2941fba83928d4c8ac399d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:39.645573 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.643225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.31.15.195:38147)
0809 16:34:39.643351 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.643353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.643361 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:39.643435 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.643436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.643456 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.643528 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:39.643643 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.644135 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.644849 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.645073 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.645125 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.645273 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.645377 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.645439 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.645476 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.645498 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260809 16:34:39.650218 2068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.648059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:39731 (local address 127.31.15.194:39469)
0809 16:34:39.648098 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.648100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.648107 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:39.648128 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.648129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.648138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.648181 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:39.648296 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.648741 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.649499 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.649765 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.649827 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.649991 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.650023 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.650067 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.650130 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.650151 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:34:39.650259 2179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.647872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:39469 (local address 127.31.15.195:39731)
0809 16:34:39.648023 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.648034 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.648083 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.648183 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.648184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.648185 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:39.648237 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.648238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.648807 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.648808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.649420 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.649422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.649831 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.649832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.649906 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.649919 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.650074 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.650076 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.650116 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.650193 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260809 16:34:39.681412 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.678909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36867 (local address 127.31.15.195:38147)
0809 16:34:39.678973 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.678975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.679147 (+ 172us) server_negotiation.cc:407] Connection header received
0809 16:34:39.679193 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.679194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.679204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.679250 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:39.679385 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.679954 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.680855 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.681052 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.681103 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.681229 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.681254 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.681284 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.681328 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.681343 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:34:39.681429 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.678845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38147 (local address 127.31.15.194:36867)
0809 16:34:39.679074 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:39.679085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:39.679148 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:39.679253 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:39.679253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:39.679255 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:39.679338 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.679340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.680034 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.680035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.680785 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:39.680788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.681113 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.681113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:39.681153 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.681170 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:39.681295 (+ 125us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:39.681296 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:34:39.681325 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 16:34:39.681364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":135,"threads_started":1}
I20260809 16:34:39.688695 1196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:39.688802 1196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:39.689455 1196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7237047554664c909687be10b01423c4: Sending DeleteTablet for 3 replicas of tablet 14f6ca0c3693430bb3181e092da3d1bc
I20260809 16:34:39.689702 1910 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45370
I20260809 16:34:39.689702 1428 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:42350
I20260809 16:34:39.689699 1668 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f6ca0c3693430bb3181e092da3d1bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:39 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:40502
I20260809 16:34:39.689932 2191 tablet_replica.cc:330] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: stopping tablet replica
I20260809 16:34:39.689993 2191 raft_consensus.cc:2229] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:39.690079 2191 raft_consensus.cc:2258] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.690353 2191 ts_tablet_manager.cc:1905] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.690572 2190 tablet_replica.cc:330] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: stopping tablet replica
I20260809 16:34:39.690632 2190 raft_consensus.cc:2229] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:39.690665 2190 raft_consensus.cc:2258] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.690860 2190 ts_tablet_manager.cc:1905] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.692169 2191 ts_tablet_manager.cc:1918] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:39.692232 2191 log.cc:1196] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/14f6ca0c3693430bb3181e092da3d1bc
I20260809 16:34:39.692444 2191 ts_tablet_manager.cc:1939] T 14f6ca0c3693430bb3181e092da3d1bc P cb10fb26750a49e2965c083ed191fa8c: Deleting consensus metadata
I20260809 16:34:39.692472 2190 ts_tablet_manager.cc:1918] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:39.692514 2190 log.cc:1196] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/14f6ca0c3693430bb3181e092da3d1bc
I20260809 16:34:39.692615 2193 tablet_replica.cc:330] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: stopping tablet replica
I20260809 16:34:39.692677 2193 raft_consensus.cc:2229] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:39.692700 2190 ts_tablet_manager.cc:1939] T 14f6ca0c3693430bb3181e092da3d1bc P 0da43d57cf2941fba83928d4c8ac399d: Deleting consensus metadata
I20260809 16:34:39.692721 2193 raft_consensus.cc:2258] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:39.692968 2193 ts_tablet_manager.cc:1905] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:39.692952 1166 catalog_manager.cc:4841] TS cb10fb26750a49e2965c083ed191fa8c (127.31.15.195:38147): tablet 14f6ca0c3693430bb3181e092da3d1bc (table test-table [id=6b22760e9f27439088253f56cfdf66e3]) successfully deleted
I20260809 16:34:39.693111 1164 catalog_manager.cc:4841] TS 0da43d57cf2941fba83928d4c8ac399d (127.31.15.193:36871): tablet 14f6ca0c3693430bb3181e092da3d1bc (table test-table [id=6b22760e9f27439088253f56cfdf66e3]) successfully deleted
I20260809 16:34:39.694147 2193 ts_tablet_manager.cc:1918] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:39.694195 2193 log.cc:1196] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/14f6ca0c3693430bb3181e092da3d1bc
I20260809 16:34:39.694371 2193 ts_tablet_manager.cc:1939] T 14f6ca0c3693430bb3181e092da3d1bc P 6135d24e35b748d8af686072967663f2: Deleting consensus metadata
I20260809 16:34:39.694702 1164 catalog_manager.cc:4841] TS 6135d24e35b748d8af686072967663f2 (127.31.15.194:39469): tablet 14f6ca0c3693430bb3181e092da3d1bc (table test-table [id=6b22760e9f27439088253f56cfdf66e3]) successfully deleted
Aug 09 16:34:39 dist-test-slave-21b9 krb5kdc[1114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293278, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:39.702661 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.690674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.31.15.193:36871)
0809 16:34:39.692833 (+ 2159us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.692835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.695479 (+ 2644us) server_negotiation.cc:407] Connection header received
0809 16:34:39.695537 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.695538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.695549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.695608 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:39.696126 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.696612 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.697273 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.697463 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.697517 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.700115 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:39.700124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:39.700124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:39.700130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:39.701583 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.701907 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.701910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.701910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.701965 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.702131 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.702132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.702133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.702269 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:39.702406 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.702481 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.702552 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2131,"thread_start_us":65,"threads_started":1}
W20260809 16:34:39.702905 1530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42354
I20260809 16:34:39.710175 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:39.703884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53358 (local address 127.31.15.254:34589)
0809 16:34:39.704029 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:39.704031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:39.704040 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:39.704063 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:39.704064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:39.704076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:39.704161 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:39.704694 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.705477 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.706145 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:39.706460 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:39.706525 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:39.707614 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:39.707616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:39.707616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:39.707627 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:39.709185 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.709485 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.709486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.709487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.709547 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:39.709707 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:39.709708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:39.709708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:39.709836 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:39.709932 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:39.709991 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 16:34:39.710071 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
W20260809 16:34:39.710419 1196 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53358
I20260809 16:34:39.711030 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1323
I20260809 16:34:39.735308 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1551
I20260809 16:34:39.745220 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1799
I20260809 16:34:39.756103 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1141
2026-08-09T16:34:39Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2691 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:39 dist-test-slave-21b9 krb5kdc[2204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:39 dist-test-slave-21b9 krb5kdc[2204](info): set up 2 sockets
Aug 09 16:34:39 dist-test-slave-21b9 krb5kdc[2204](info): commencing operation
krb5kdc: starting...
W20260809 16:34:40.905200 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:40 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:41.019390 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32885
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:32885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:41.100325 2224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:41.100529 2224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:41.100558 2224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:41.101665 2224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:41.101703 2224 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:34:41.101720 2224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:41.101737 2224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:41.101752 2224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:41.101768 2224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:41.103190 2224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:32885
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:41.103444 2224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:41.103664 2224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:41.108193 2224 server_base.cc:1034] running on GCE node
W20260809 16:34:41.108165 2234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.108150 2231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.108129 2230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:41.108590 2224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:41.108870 2224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:41.110033 2224 hybrid_clock.cc:648] HybridClock initialized: now 1786293281110011 us; error 34 us; skew 500 ppm
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:41.112636 2224 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:41.113040 2224 webserver.cc:460] Webserver started at http://127.31.15.254:36171/ using document root <none> and password file <none>
I20260809 16:34:41.113255 2224 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:41.113302 2224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:41.113425 2224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:41.114436 2224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "abc62ad43ca145c1bf05d0a291dd2c13"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4f3de618c3b1f3bac120ce54f79540a4"
server_key_iv: "c5248f310dd5f3022e9de58124622ab5"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.114768 2224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "abc62ad43ca145c1bf05d0a291dd2c13"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4f3de618c3b1f3bac120ce54f79540a4"
server_key_iv: "c5248f310dd5f3022e9de58124622ab5"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.116063 2224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:41.116691 2246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.116859 2224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:41.116933 2224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "abc62ad43ca145c1bf05d0a291dd2c13"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4f3de618c3b1f3bac120ce54f79540a4"
server_key_iv: "c5248f310dd5f3022e9de58124622ab5"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.117004 2224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:41.135095 2224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:41.136817 2224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:41.136926 2224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:41.141508 2224 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:32885
I20260809 16:34:41.141568 2350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:32885 every 8 connection(s)
I20260809 16:34:41.141851 2224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:41.142490 2353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.144449 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2224
I20260809 16:34:41.144542 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:41.144726 31807 external_mini_cluster.cc:1428] Setting key 6517cc32e99bd990eb0ae47eddbf6a8e
I20260809 16:34:41.144961 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Bootstrap starting.
I20260809 16:34:41.145678 2353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.145936 2353 log.cc:824] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:41.146567 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: No bootstrap required, opened a new log
I20260809 16:34:41.148980 2353 raft_consensus.cc:348] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } }
I20260809 16:34:41.149168 2353 raft_consensus.cc:374] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.149199 2353 raft_consensus.cc:729] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abc62ad43ca145c1bf05d0a291dd2c13, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.149322 2353 consensus_queue.cc:260] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } }
I20260809 16:34:41.149396 2353 raft_consensus.cc:388] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:41.149430 2353 raft_consensus.cc:482] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:41.149471 2353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.150085 2353 raft_consensus.cc:504] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } }
I20260809 16:34:41.150189 2353 leader_election.cc:302] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abc62ad43ca145c1bf05d0a291dd2c13; no voters:
I20260809 16:34:41.150354 2353 leader_election.cc:288] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:41.150410 2362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:41.150586 2362 raft_consensus.cc:686] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [term 1 LEADER]: Becoming Leader. State: Replica: abc62ad43ca145c1bf05d0a291dd2c13, State: Running, Role: LEADER
I20260809 16:34:41.150606 2353 sys_catalog.cc:564] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:41.150763 2362 consensus_queue.cc:237] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } }
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.151865 2363 sys_catalog.cc:455] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abc62ad43ca145c1bf05d0a291dd2c13. Latest consensus state: current_term: 1 leader_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } } }
I20260809 16:34:41.151947 2363 sys_catalog.cc:458] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:41.152110 2364 sys_catalog.cc:455] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abc62ad43ca145c1bf05d0a291dd2c13" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32885 } } }
I20260809 16:34:41.152225 2364 sys_catalog.cc:458] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:41.152235 2387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:41.152724 2387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:41.154316 2387 catalog_manager.cc:1347] Generated new cluster ID: 6bc8abd5fbeb4cbbbc3cc61cdcf8a703
I20260809 16:34:41.154372 2387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:41.155897 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.145274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.31.15.254:32885)
0809 16:34:41.145453 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.145463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.145487 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:34:41.145960 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.145967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.146018 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.146157 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.146755 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.148263 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.148977 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.149574 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.149677 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.152575 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.152579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.152579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.152591 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.154474 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.154790 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.154793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.154793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.154881 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.155056 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.155057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.155057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.155309 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.155470 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.155555 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.155685 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260809 16:34:41.161079 2387 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:41.161757 2387 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:41.169651 2387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Generated new TSK 0
I20260809 16:34:41.169811 2387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:41.198827 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32885
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:41.278755 2395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:41.278949 2395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:41.278981 2395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:41.280089 2395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:41.280130 2395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:41.280157 2395 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:34:41.280171 2395 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:41.280194 2395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:41.281625 2395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:41.281857 2395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:41.282064 2395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:41.286655 2403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.286702 2406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.286731 2404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:41.286888 2395 server_base.cc:1034] running on GCE node
I20260809 16:34:41.287132 2395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:41.287384 2395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:41.288547 2395 hybrid_clock.cc:648] HybridClock initialized: now 1786293281288532 us; error 30 us; skew 500 ppm
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:41.290871 2395 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:41.291316 2395 webserver.cc:460] Webserver started at http://127.31.15.193:46661/ using document root <none> and password file <none>
I20260809 16:34:41.291543 2395 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:41.291587 2395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:41.291712 2395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:41.292706 2395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "1b4222f2b0d242c1ba3a3d10687970af"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "d1821ff42e0ae0b11e41be9a186a879a"
server_key_iv: "1fa4e548dda39a27cdc33a9cb7f8dda9"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.293015 2395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "1b4222f2b0d242c1ba3a3d10687970af"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "d1821ff42e0ae0b11e41be9a186a879a"
server_key_iv: "1fa4e548dda39a27cdc33a9cb7f8dda9"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.294188 2395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:41.294826 2416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.294958 2395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:41.295039 2395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "1b4222f2b0d242c1ba3a3d10687970af"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "d1821ff42e0ae0b11e41be9a186a879a"
server_key_iv: "1fa4e548dda39a27cdc33a9cb7f8dda9"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.295102 2395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:41.302592 2395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:41.304267 2395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:41.304392 2395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:41.304863 2395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:41.304905 2395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.304936 2395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:41.304958 2395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.311950 2395 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:33843
I20260809 16:34:41.312248 2599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:33843 every 8 connection(s)
I20260809 16:34:41.312342 2395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:41.313419 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2395
I20260809 16:34:41.313531 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:41.313723 31807 external_mini_cluster.cc:1428] Setting key fba835de0420ca9b346b94b03240adb0
I20260809 16:34:41.314643 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.305337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:50861 (local address 127.31.15.254:32885)
0809 16:34:41.305412 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.305415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.305605 (+ 190us) server_negotiation.cc:407] Connection header received
0809 16:34:41.306236 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.306238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.306252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.306308 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.307136 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.307743 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.309177 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.309412 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.309477 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.311687 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.311689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.311690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.311698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.313157 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.313661 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.313662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.313662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.313718 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.314000 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.314001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.314002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.314136 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.314199 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.314451 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.314551 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:34:41.314769 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.305305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.193:50861)
0809 16:34:41.305470 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.305514 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.306109 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.306414 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.306418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.306756 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.307046 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.307053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.307823 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.307824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.309086 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.309090 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.309527 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.309529 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.309654 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.310132 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.310141 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.311588 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.313243 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.313252 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.313254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.313584 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.313781 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.313782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.313782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.313920 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.314200 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.314208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.314332 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.314419 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.314564 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 16:34:41.315243 2602 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32885
I20260809 16:34:41.315351 2602 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:41.315531 2602 heartbeater.cc:511] Master 127.31.15.254:32885 requested a full tablet report, sending...
I20260809 16:34:41.316072 2273 ts_manager.cc:194] Registered new tserver with Master: 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843)
I20260809 16:34:41.316923 2273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:50861
I20260809 16:34:41.321975 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.316335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.193:35187)
0809 16:34:41.316522 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.316533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.316591 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.316702 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.316703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.316879 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.316933 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.316935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.317777 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.317778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.318277 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.318278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.318698 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.318699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.318758 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.319134 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.319138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.319532 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.321064 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.321065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.321065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.321217 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.321431 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.321432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.321432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.321498 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.321748 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.321748 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.321785 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.321830 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.321911 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":119,"threads_started":1}
I20260809 16:34:41.322137 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.316344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35187 (local address 127.31.15.254:32885)
0809 16:34:41.316388 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.316390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.316582 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:34:41.316636 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.316637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.316647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.316700 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.317002 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.317691 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.318355 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.318626 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.318694 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.319606 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.319607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.319608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.319615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.320974 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.321279 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.321280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.321280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.321338 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.321564 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.321565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.321565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.321689 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.321750 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.321880 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.321956 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:41.339177 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32885
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:41.420092 2622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:41.420296 2622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:41.420320 2622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:41.421403 2622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:41.421439 2622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:41.421459 2622 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:34:41.421469 2622 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:41.421484 2622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:41.422879 2622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:41.423066 2622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:41.423254 2622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:41.427949 2635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.427963 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.427974 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:41.428233 2622 server_base.cc:1034] running on GCE node
I20260809 16:34:41.428399 2622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:41.428587 2622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:41.429713 2622 hybrid_clock.cc:648] HybridClock initialized: now 1786293281429704 us; error 24 us; skew 500 ppm
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:41.432112 2622 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:41.432543 2622 webserver.cc:460] Webserver started at http://127.31.15.194:35489/ using document root <none> and password file <none>
I20260809 16:34:41.432740 2622 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:41.432787 2622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:41.432914 2622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:41.433902 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "675f61c968e44d669f5d691361d778f7"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4192aca61478f84fd866c1298802ca96"
server_key_iv: "c10dd0237e027bd362d8d586a5a18a8e"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.434237 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "675f61c968e44d669f5d691361d778f7"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4192aca61478f84fd866c1298802ca96"
server_key_iv: "c10dd0237e027bd362d8d586a5a18a8e"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.435483 2622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:41.436141 2644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.436285 2622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.436367 2622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "675f61c968e44d669f5d691361d778f7"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "4192aca61478f84fd866c1298802ca96"
server_key_iv: "c10dd0237e027bd362d8d586a5a18a8e"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.436424 2622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:41.448575 2622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:41.450313 2622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:41.450443 2622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:41.450978 2622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:41.451027 2622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.451056 2622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:41.451079 2622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.457947 2622 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:34809
I20260809 16:34:41.458364 2622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:41.458339 2845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:34809 every 8 connection(s)
I20260809 16:34:41.460237 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.451402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:53805 (local address 127.31.15.254:32885)
0809 16:34:41.451475 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.451477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.451704 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:34:41.452242 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.452243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.452257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.452308 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.452956 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.453448 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.454435 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.454634 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.454690 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.457530 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.457531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.457532 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.457539 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.458988 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.459378 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.459379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.459380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.459439 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.459652 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.459653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.459653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.459764 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.459822 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.460070 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.460148 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:41.460395 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.451435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.194:53805)
0809 16:34:41.451589 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.451624 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.452148 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.452376 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.452380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.452675 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.452860 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.452865 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.453579 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.453580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.454354 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.454356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.454702 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.454703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.454840 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.455339 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.455350 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.457427 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.459059 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.459066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.459068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.459318 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.459495 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.459496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.459496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.459597 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.459818 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.459825 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.459956 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.460044 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.460196 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260809 16:34:41.460827 2847 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32885
I20260809 16:34:41.460911 2847 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:41.461089 2847 heartbeater.cc:511] Master 127.31.15.254:32885 requested a full tablet report, sending...
I20260809 16:34:41.461457 2274 ts_manager.cc:194] Registered new tserver with Master: 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809)
I20260809 16:34:41.461850 2274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:53805
I20260809 16:34:41.463824 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2622
I20260809 16:34:41.463922 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:41.464110 31807 external_mini_cluster.cc:1428] Setting key 6bb8868c3e52d265f24ceb03a228e0bc
I20260809 16:34:41.467911 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.461793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.194:41405)
0809 16:34:41.461945 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.461957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.462026 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.462129 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.462130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.462297 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.462347 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.462348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.463011 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.463012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.463599 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.463601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.464067 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.464068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.464146 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.464564 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.464568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.464857 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.466933 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.466934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.466934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.467106 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.467337 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.467338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.467338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.467392 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.467692 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.467692 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.467727 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.467775 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.467849 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260809 16:34:41.467955 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.461868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:41405 (local address 127.31.15.254:32885)
0809 16:34:41.461904 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.461905 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.462013 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:34:41.462071 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.462071 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.462080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.462115 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.462410 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.462924 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.463674 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.463938 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.464014 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.464925 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.464926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.464927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.464933 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.466836 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.467169 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.467170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.467170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.467229 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.467443 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.467443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.467444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.467633 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.467703 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.467807 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.467876 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:41.488073 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32885
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:41.568184 2862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:41.568369 2862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:41.568400 2862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:41.569515 2862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:41.569557 2862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:41.569584 2862 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 16:34:41.569599 2862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:34:41.569621 2862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:41.571071 2862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:32885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:41.571329 2862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:41.571540 2862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:41.576153 2877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.576112 2869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:41.576125 2871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:41.576313 2862 server_base.cc:1034] running on GCE node
I20260809 16:34:41.576483 2862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:41.576674 2862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:41.577823 2862 hybrid_clock.cc:648] HybridClock initialized: now 1786293281577805 us; error 29 us; skew 500 ppm
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:41.580216 2862 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:41.580621 2862 webserver.cc:460] Webserver started at http://127.31.15.195:46359/ using document root <none> and password file <none>
I20260809 16:34:41.580837 2862 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:41.580885 2862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:41.581005 2862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:41.581985 2862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "906085f716b6467ba8fbb230ee00280a"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "7a504c2ff62bd9f89d654ae7705e40cc"
server_key_iv: "488c1fb12a7c609e04d4cc9ef307fefa"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.582391 2862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "906085f716b6467ba8fbb230ee00280a"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "7a504c2ff62bd9f89d654ae7705e40cc"
server_key_iv: "488c1fb12a7c609e04d4cc9ef307fefa"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.583696 2862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:41.584342 2886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.584533 2862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:41.584620 2862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "906085f716b6467ba8fbb230ee00280a"
format_stamp: "Formatted at 2026-08-09 16:34:41 on dist-test-slave-21b9"
server_key: "7a504c2ff62bd9f89d654ae7705e40cc"
server_key_iv: "488c1fb12a7c609e04d4cc9ef307fefa"
server_key_version: "encryptionkey@0"
I20260809 16:34:41.584672 2862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:41.629489 2862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:41.631649 2862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:41.631763 2862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:41.632241 2862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:41.632282 2862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.632306 2862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:41.632319 2862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.638684 2862 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:39277
I20260809 16:34:41.638794 3077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:39277 every 8 connection(s)
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.639078 2862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:41.641640 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.632730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:47215 (local address 127.31.15.254:32885)
0809 16:34:41.632843 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.632845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.633039 (+ 194us) server_negotiation.cc:407] Connection header received
0809 16:34:41.633551 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.633552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.633565 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.633643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.634458 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.635213 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.636730 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.637032 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.637104 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.639528 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.639529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.639530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.639537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.640653 (+ 1116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.640981 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.640982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.640982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.641029 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.641199 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.641199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.641200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.641293 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.641337 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.641523 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.641579 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 16:34:41.641860 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.632702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.195:47215)
0809 16:34:41.632910 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.632950 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.633438 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.633739 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.633744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.634084 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.634358 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.634365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.635375 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.635375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.636642 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.636645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.637111 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.637112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.637215 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.637686 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.637697 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.639445 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.640716 (+ 1271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.640721 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.640723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.640924 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.641072 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.641072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.641072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.641158 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.641333 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.641335 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.641439 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.641511 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.641643 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260809 16:34:41.642287 3078 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32885
I20260809 16:34:41.642366 3078 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:41.642532 3078 heartbeater.cc:511] Master 127.31.15.254:32885 requested a full tablet report, sending...
I20260809 16:34:41.642894 2273 ts_manager.cc:194] Registered new tserver with Master: 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:41.642948 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2862
I20260809 16:34:41.643056 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:41.643246 31807 external_mini_cluster.cc:1428] Setting key 507a6605dc01f3d2b74f60cd5a746ae6
I20260809 16:34:41.643409 2273 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:47215
I20260809 16:34:41.644073 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:34:41.648700 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.643329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:59277 (local address 127.31.15.254:32885)
0809 16:34:41.643376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.643378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.643552 (+ 174us) server_negotiation.cc:407] Connection header received
0809 16:34:41.643606 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.643607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.643617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.643666 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.644028 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.644635 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.645468 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.645640 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.645681 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.646539 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.646541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.646541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.646548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.647775 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.648108 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.648108 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.648108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.648152 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.648305 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.648306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.648306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.648388 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.648430 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.648592 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.648644 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:34:41.648759 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.643242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.31.15.195:59277)
0809 16:34:41.643496 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.643506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.643563 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.643738 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.643739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.643887 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.643934 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.643935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.644732 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.644733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.645398 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.645400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.645694 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.645695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.645768 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.646161 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.646166 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.646461 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.647848 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.647849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.647849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.648047 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.648199 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.648199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.648200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.648259 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.648443 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.648444 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.648495 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.648573 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.648669 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":179,"threads_started":1}
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.667011 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.660971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46668 (local address 127.31.15.254:32885)
0809 16:34:41.661021 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.661022 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.661050 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:34:41.661071 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.661072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.661081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.661117 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.661604 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.662146 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.662878 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.663061 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.663108 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.665139 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.665140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.665140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.665146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.666192 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.666482 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.666482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.666483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.666527 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.666669 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.666669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.666670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.666758 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.666821 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.666891 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.666945 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:34:41.668684 2274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:41.670437 2274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:41.676585 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.0.0.1:60628)
0809 16:34:41.673100 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.673107 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.673141 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.673300 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.673303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.673305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.673372 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.673374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674347 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.674348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.675609 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.675612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676306 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.676383 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676396 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.676459 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.676511 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":100,"threads_started":1}
I20260809 16:34:41.676615 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.31.15.193:33843)
0809 16:34:41.673032 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.673035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.673151 (+ 116us) server_negotiation.cc:407] Connection header received
0809 16:34:41.673185 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.673187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.673200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.673267 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.673461 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674276 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.675699 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676233 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676296 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676356 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.676479 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.676516 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260809 16:34:41.676949 3110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34809 (local address 127.0.0.1:34920)
0809 16:34:41.672891 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.672936 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.673007 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.673308 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.673310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.673312 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.673389 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.673391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674624 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.674626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.675746 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.675749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676686 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.676754 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676760 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.676811 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.676858 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":17,"thread_start_us":45,"threads_started":1}
I20260809 16:34:41.676960 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.31.15.194:34809)
0809 16:34:41.672887 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.672892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.673179 (+ 287us) server_negotiation.cc:407] Connection header received
0809 16:34:41.673215 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.673217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.673229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.673296 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.673475 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674543 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.675824 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676525 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676594 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676648 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.676828 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.676870 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260809 16:34:41.676949 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39277 (local address 127.0.0.1:53018)
0809 16:34:41.673373 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.673382 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.673425 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.673665 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.673666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.673668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.673712 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.673713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674843 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.674844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.675729 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.675732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676685 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.676754 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676760 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.676815 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.676858 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":7,"thread_start_us":103,"threads_started":1}
I20260809 16:34:41.677348 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.672852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53018 (local address 127.31.15.195:39277)
0809 16:34:41.673005 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.673009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.673478 (+ 469us) server_negotiation.cc:407] Connection header received
0809 16:34:41.673521 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.673524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.673540 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.673603 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.673790 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.674775 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.675806 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.676525 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.676595 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.676649 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.677229 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.677274 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260809 16:34:41.677486 2500 tablet_service.cc:1467] Processing CreateTablet for tablet c85fa45cb4cf4ababc22917538bef553 (DEFAULT_TABLE table=test-table [id=3170fbb944344a368dabbeda65d0fc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:41.677695 2720 tablet_service.cc:1467] Processing CreateTablet for tablet c85fa45cb4cf4ababc22917538bef553 (DEFAULT_TABLE table=test-table [id=3170fbb944344a368dabbeda65d0fc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:41.677835 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85fa45cb4cf4ababc22917538bef553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.678011 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85fa45cb4cf4ababc22917538bef553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.680588 3116 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Bootstrap starting.
I20260809 16:34:41.680815 3117 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Bootstrap starting.
I20260809 16:34:41.681136 2966 tablet_service.cc:1467] Processing CreateTablet for tablet c85fa45cb4cf4ababc22917538bef553 (DEFAULT_TABLE table=test-table [id=3170fbb944344a368dabbeda65d0fc25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:41.681404 3117 tablet_bootstrap.cc:654] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.681423 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c85fa45cb4cf4ababc22917538bef553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.681679 3117 log.cc:824] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:41.682397 3116 tablet_bootstrap.cc:654] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.682410 3117 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: No bootstrap required, opened a new log
I20260809 16:34:41.682471 3117 ts_tablet_manager.cc:1397] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:41.682675 3116 log.cc:824] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:41.683331 3116 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: No bootstrap required, opened a new log
I20260809 16:34:41.683395 3116 ts_tablet_manager.cc:1397] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:34:41.684377 3117 raft_consensus.cc:348] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.684608 3124 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Bootstrap starting.
I20260809 16:34:41.684661 3117 raft_consensus.cc:374] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.684790 3117 raft_consensus.cc:729] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.684937 3117 consensus_queue.cc:260] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.684933 3116 raft_consensus.cc:348] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.685110 3116 raft_consensus.cc:374] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.685153 3116 raft_consensus.cc:729] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.685195 3117 ts_tablet_manager.cc:1428] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:34:41.685266 3116 consensus_queue.cc:260] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.685314 2602 heartbeater.cc:503] Master 127.31.15.254:32885 was elected leader, sending a full tablet report...
I20260809 16:34:41.685514 3116 ts_tablet_manager.cc:1428] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:34:41.685541 2847 heartbeater.cc:503] Master 127.31.15.254:32885 was elected leader, sending a full tablet report...
I20260809 16:34:41.685842 3124 tablet_bootstrap.cc:654] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.686173 3124 log.cc:824] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:41.686856 3124 tablet_bootstrap.cc:492] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: No bootstrap required, opened a new log
I20260809 16:34:41.686911 3124 ts_tablet_manager.cc:1397] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:34:41.688397 3124 raft_consensus.cc:348] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.688511 3124 raft_consensus.cc:374] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.688541 3124 raft_consensus.cc:729] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.688640 3124 consensus_queue.cc:260] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.688850 3124 ts_tablet_manager.cc:1428] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:34:41.688915 3078 heartbeater.cc:503] Master 127.31.15.254:32885 was elected leader, sending a full tablet report...
I20260809 16:34:41.697909 3131 raft_consensus.cc:482] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:41.697984 3131 raft_consensus.cc:504] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.698262 3131 leader_election.cc:288] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843)
I20260809 16:34:41.700757 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.698402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34809 (local address 127.31.15.195:37043)
0809 16:34:41.698452 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.698462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.698500 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.698667 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.698668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.698670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.698722 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.698723 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.699316 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.699317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.700048 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.700050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.700543 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.700610 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.700612 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.700655 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.700691 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":33}
I20260809 16:34:41.700754 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.698446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:37043 (local address 127.31.15.194:34809)
0809 16:34:41.698486 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.698488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.698497 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:41.698570 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.698571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.698596 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.698640 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.698774 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.699225 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700122 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.700466 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700527 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.700535 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.700666 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.700697 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:41.700870 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.698479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:52907 (local address 127.31.15.193:33843)
0809 16:34:41.698514 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.698515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.698632 (+ 117us) server_negotiation.cc:407] Connection header received
0809 16:34:41.698687 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.698688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.698697 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.698742 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.698871 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.699363 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700208 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.700573 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700633 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.700641 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.700778 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.700811 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:41.700934 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.698407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.31.15.195:52907)
0809 16:34:41.698563 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.698572 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.698630 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.698749 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.698749 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.698751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.698818 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.698820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.699430 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.699431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.700134 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.700136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.700646 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.700647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.700721 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.700724 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.700767 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.700802 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":10,"thread_start_us":84,"threads_started":1}
I20260809 16:34:41.701053 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85fa45cb4cf4ababc22917538bef553" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7" is_pre_election: true
I20260809 16:34:41.701181 2751 raft_consensus.cc:2454] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 0.
I20260809 16:34:41.701192 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85fa45cb4cf4ababc22917538bef553" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af" is_pre_election: true
I20260809 16:34:41.701313 2533 raft_consensus.cc:2454] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 0.
I20260809 16:34:41.701385 2890 leader_election.cc:302] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 675f61c968e44d669f5d691361d778f7, 906085f716b6467ba8fbb230ee00280a; no voters:
I20260809 16:34:41.701502 3131 raft_consensus.cc:2781] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:41.701556 3131 raft_consensus.cc:482] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:41.701582 3131 raft_consensus.cc:3037] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.702194 3131 raft_consensus.cc:504] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.702306 3131 leader_election.cc:288] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 election: Requested vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843)
I20260809 16:34:41.702442 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85fa45cb4cf4ababc22917538bef553" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7"
I20260809 16:34:41.702445 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c85fa45cb4cf4ababc22917538bef553" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af"
I20260809 16:34:41.702502 2751 raft_consensus.cc:3037] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.702510 2533 raft_consensus.cc:3037] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.703112 2751 raft_consensus.cc:2454] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 1.
I20260809 16:34:41.703145 2533 raft_consensus.cc:2454] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 1.
I20260809 16:34:41.703256 2890 leader_election.cc:302] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 675f61c968e44d669f5d691361d778f7, 906085f716b6467ba8fbb230ee00280a; no voters:
I20260809 16:34:41.703392 3131 raft_consensus.cc:2781] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:41.703523 3131 raft_consensus.cc:686] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 1 LEADER]: Becoming Leader. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Running, Role: LEADER
I20260809 16:34:41.703619 3131 consensus_queue.cc:237] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } }
I20260809 16:34:41.704360 2274 catalog_manager.cc:5495] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a reported cstate change: term changed from 0 to 1, leader changed from <none> to 906085f716b6467ba8fbb230ee00280a (127.31.15.195). New cstate: current_term: 1 leader_uuid: "906085f716b6467ba8fbb230ee00280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } health_report { overall_health: UNKNOWN } } }
W20260809 16:34:41.708861 2849 tablet.cc:2369] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 16:34:41 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293281, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:41.737737 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.731873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32885 (local address 127.0.0.1:46684)
0809 16:34:41.732027 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.732037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.732100 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.732216 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.732216 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.732513 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:41.732560 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.732561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.733146 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.733147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.733659 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.733660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.733915 (+ 255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.733916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.733960 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.734375 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:41.734380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:41.735691 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:41.736898 (+ 1207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.736906 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.736908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.737101 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.737246 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:41.737247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:41.737247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:41.737302 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:41.737468 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:41.737473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:41.737514 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.737561 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.737646 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260809 16:34:41.737788 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.731951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.31.15.254:32885)
0809 16:34:41.732027 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.732029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.732093 (+ 64us) server_negotiation.cc:407] Connection header received
0809 16:34:41.732145 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.732146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.732154 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.732202 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:41.732619 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.733093 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.733709 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.733872 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.733912 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.735765 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:41.735767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:41.735767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:41.735773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:41.736824 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.737158 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.737159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.737159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.737201 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:41.737342 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:41.737342 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:41.737343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:41.737428 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:41.737470 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.737588 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.737689 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":55}
I20260809 16:34:41.739719 2274 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:46684:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:34:41.739920 2274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:41.741263 2966 tablet_service.cc:1467] Processing CreateTablet for tablet bc853d5d54904f21bf784d103e22147d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08e4196af1d4400787aa595492419928]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:41.741246 2720 tablet_service.cc:1467] Processing CreateTablet for tablet bc853d5d54904f21bf784d103e22147d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08e4196af1d4400787aa595492419928]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:41.741266 2500 tablet_service.cc:1467] Processing CreateTablet for tablet bc853d5d54904f21bf784d103e22147d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=08e4196af1d4400787aa595492419928]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:34:41.741415 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc853d5d54904f21bf784d103e22147d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.741415 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc853d5d54904f21bf784d103e22147d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.741415 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc853d5d54904f21bf784d103e22147d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:41.742820 3124 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: Bootstrap starting.
I20260809 16:34:41.742892 3116 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: Bootstrap starting.
I20260809 16:34:41.743251 3124 tablet_bootstrap.cc:654] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.743499 3116 tablet_bootstrap.cc:654] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.743844 3117 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: Bootstrap starting.
I20260809 16:34:41.743937 3124 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: No bootstrap required, opened a new log
I20260809 16:34:41.743992 3124 ts_tablet_manager.cc:1397] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:41.744148 3124 raft_consensus.cc:348] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.744205 3124 raft_consensus.cc:374] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.744217 3124 raft_consensus.cc:729] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.744251 3124 consensus_queue.cc:260] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.744335 3124 ts_tablet_manager.cc:1428] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.744383 3131 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.744447 3131 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.744457 3117 tablet_bootstrap.cc:654] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:41.744993 3116 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: No bootstrap required, opened a new log
I20260809 16:34:41.745048 3116 ts_tablet_manager.cc:1397] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:34:41.745150 3117 tablet_bootstrap.cc:492] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: No bootstrap required, opened a new log
I20260809 16:34:41.745199 3117 ts_tablet_manager.cc:1397] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:41.745198 3116 raft_consensus.cc:348] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.745271 3116 raft_consensus.cc:374] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.745292 3116 raft_consensus.cc:729] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.745332 3116 consensus_queue.cc:260] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.745347 3117 raft_consensus.cc:348] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.745415 3116 ts_tablet_manager.cc:1428] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.745419 3117 raft_consensus.cc:374] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:41.745448 3117 raft_consensus.cc:729] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Initialized, Role: FOLLOWER
I20260809 16:34:41.745466 3127 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.745496 3117 consensus_queue.cc:260] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.745541 3127 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.745572 3117 ts_tablet_manager.cc:1428] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:41.745663 3125 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.745754 3125 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 16:34:41.813117 2605 tablet.cc:2369] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:34:41.889653 3080 tablet.cc:2369] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:41.942341 3125 raft_consensus.cc:482] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:41.942447 3125 raft_consensus.cc:504] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.942785 3125 leader_election.cc:288] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:41.946513 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.943039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35757 (local address 127.31.15.195:39277)
0809 16:34:41.943079 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.943081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.943147 (+ 66us) server_negotiation.cc:407] Connection header received
0809 16:34:41.943209 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.943210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.943219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.943304 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.943458 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.944209 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.945637 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.946188 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.946241 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.946248 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.946410 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.946445 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:34:41.946535 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.943022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:56579 (local address 127.31.15.194:34809)
0809 16:34:41.943070 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.943072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.943149 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:34:41.943209 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.943210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.943219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.943307 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:41.943457 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.944214 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.945637 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.946168 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.946238 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.946248 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.946411 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.946446 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:34:41.946550 3151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.942944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39277 (local address 127.31.15.193:35757)
0809 16:34:41.943079 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.943087 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.943152 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.943329 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.943329 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.943331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.943398 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.943399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.944284 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.944284 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.945557 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.945560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.946253 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.946253 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.946345 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.946350 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.946400 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.946446 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":63,"thread_start_us":68,"threads_started":1}
I20260809 16:34:41.946568 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.942944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34809 (local address 127.31.15.193:56579)
0809 16:34:41.943075 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.943086 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.943152 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.943315 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.943317 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.943319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:41.943382 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.943385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.944283 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.944284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.945571 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.945573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.946249 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.946250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.946345 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.946352 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.946400 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.946446 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":33,"thread_start_us":67,"threads_started":1}
I20260809 16:34:41.946836 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc853d5d54904f21bf784d103e22147d" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7" is_pre_election: true
I20260809 16:34:41.946918 2751 raft_consensus.cc:2454] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 0.
I20260809 16:34:41.947057 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc853d5d54904f21bf784d103e22147d" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a" is_pre_election: true
I20260809 16:34:41.947098 2421 leader_election.cc:302] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 675f61c968e44d669f5d691361d778f7; no voters:
I20260809 16:34:41.947173 2996 raft_consensus.cc:2454] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 0.
I20260809 16:34:41.947222 3125 raft_consensus.cc:2781] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:41.947288 3125 raft_consensus.cc:482] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:41.947312 3125 raft_consensus.cc:3037] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.948017 3125 raft_consensus.cc:504] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.948137 3125 leader_election.cc:288] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 election: Requested vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:41.948282 2751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc853d5d54904f21bf784d103e22147d" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7"
I20260809 16:34:41.948356 2751 raft_consensus.cc:3037] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.948341 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc853d5d54904f21bf784d103e22147d" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a"
I20260809 16:34:41.948412 2996 raft_consensus.cc:3037] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:41.948935 2751 raft_consensus.cc:2454] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 1.
I20260809 16:34:41.949096 2421 leader_election.cc:302] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 675f61c968e44d669f5d691361d778f7; no voters:
I20260809 16:34:41.949096 2996 raft_consensus.cc:2454] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 1.
I20260809 16:34:41.949271 3125 raft_consensus.cc:2781] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:41.949426 3125 raft_consensus.cc:686] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [term 1 LEADER]: Becoming Leader. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Running, Role: LEADER
I20260809 16:34:41.949517 3125 consensus_queue.cc:237] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:41.949925 3153 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: TxnStatusTablet state changed. Reason: New leader 1b4222f2b0d242c1ba3a3d10687970af. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.950055 3153 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:41.950266 3155 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:34:41.950399 3155 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:34:41.950330 2273 catalog_manager.cc:5495] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193). New cstate: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } health_report { overall_health: UNKNOWN } } }
I20260809 16:34:41.970901 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.968126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.31.15.193:33843)
0809 16:34:41.968171 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.968173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.968185 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:34:41.968272 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.968273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.968290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.968354 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:41.968510 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.968982 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.970102 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.970294 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.970354 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.970581 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.970688 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.970762 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.970807 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.970830 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:41.970952 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.968047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.0.0.1:60630)
0809 16:34:41.968134 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.968147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.968192 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.968367 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.968367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.968369 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:41.968423 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.968425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.969050 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.969050 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.970003 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.970006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.970364 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.970366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.970432 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.970451 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.970755 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.970761 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.970805 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.970855 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 16:34:41.973199 2996 raft_consensus.cc:1261] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Refusing update from remote peer 1b4222f2b0d242c1ba3a3d10687970af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:41.973202 2751 raft_consensus.cc:1261] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Refusing update from remote peer 1b4222f2b0d242c1ba3a3d10687970af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:41.973516 3153 consensus_queue.cc:1035] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Connected to new peer: Peer: permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:41.973649 3125 consensus_queue.cc:1035] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Connected to new peer: Peer: permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:41.974812 3131 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: TxnStatusTablet state changed. Reason: New leader 1b4222f2b0d242c1ba3a3d10687970af. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.974906 3131 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.974871 3127 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: TxnStatusTablet state changed. Reason: New leader 1b4222f2b0d242c1ba3a3d10687970af. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.974972 3127 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.975963 3125 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.976050 3125 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:41.976118 3160 mvcc.cc:205] Tried to move back new op lower bound from 7316657282959589376 to 7316657282865840128. Current Snapshot: MvccSnapshot[applied={T|T < 7316657282959589376}]
I20260809 16:34:41.976114 3127 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.976189 3127 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 675f61c968e44d669f5d691361d778f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.976356 3125 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.976441 3125 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 1b4222f2b0d242c1ba3a3d10687970af: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:34:41.976841 3163 mvcc.cc:205] Tried to move back new op lower bound from 7316657282959589376 to 7316657282865840128. Current Snapshot: MvccSnapshot[applied={T|T < 7316657282959589376}]
I20260809 16:34:41.976950 3131 tablet_replica.cc:439] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } } }
I20260809 16:34:41.977030 3131 tablet_replica.cc:442] T bc853d5d54904f21bf784d103e22147d P 906085f716b6467ba8fbb230ee00280a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:34:41.977092 3164 mvcc.cc:205] Tried to move back new op lower bound from 7316657282959589376 to 7316657282865840128. Current Snapshot: MvccSnapshot[applied={T|T < 7316657282959589376}]
I20260809 16:34:41.981798 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.979164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.31.15.195:39277)
0809 16:34:41.979219 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.979221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.979228 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:41.979256 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.979257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.979293 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.979359 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:41.979504 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.980248 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.980982 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.981278 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.981343 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.981484 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.981600 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.981670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.981691 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.981729 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:34:41.986688 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.984165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:51939 (local address 127.31.15.193:33843)
0809 16:34:41.984204 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.984206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.984216 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:41.984288 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.984289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.984297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.984335 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:41.984451 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.984893 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.986051 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.986237 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.986299 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.986489 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.986517 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.986559 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.986619 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.986638 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:41.986730 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.984092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.31.15.195:51939)
0809 16:34:41.984168 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.984180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.984224 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.984347 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.984347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.984348 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:41.984405 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.984406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.984954 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.984955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.985951 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.985953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.986307 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.986308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.986392 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.986407 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.986569 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.986571 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.986616 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.986658 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:34:41.989094 2751 raft_consensus.cc:1261] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Refusing update from remote peer 906085f716b6467ba8fbb230ee00280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:41.989185 2533 raft_consensus.cc:1261] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Refusing update from remote peer 906085f716b6467ba8fbb230ee00280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:41.989429 3133 consensus_queue.cc:1035] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:41.989571 3131 consensus_queue.cc:1035] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:41.997958 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.995088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35663 (local address 127.31.15.195:39277)
0809 16:34:41.995129 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:41.995131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:41.995288 (+ 157us) server_negotiation.cc:407] Connection header received
0809 16:34:41.995374 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:41.995375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:41.995384 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:41.995437 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:41.995554 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.996266 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.997218 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.997506 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.997565 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.997725 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.997775 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.997826 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:41.997875 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:34:41.997894 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:41.997972 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:41.995039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39277 (local address 127.31.15.193:35663)
0809 16:34:41.995205 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:41.995216 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:41.995310 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:41.995440 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:41.995441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:41.995443 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:41.995499 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.995500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.996336 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.996337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.997062 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:41.997065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:41.997585 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:41.997586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:41.997642 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:41.997657 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:41.997827 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:41.997829 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:41.997870 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:34:41.997907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260809 16:34:42.010268 2273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:42.010419 2273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:42.011335 2273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Sending DeleteTablet for 3 replicas of tablet c85fa45cb4cf4ababc22917538bef553
I20260809 16:34:42.011966 2720 tablet_service.cc:1514] Processing DeleteTablet for tablet c85fa45cb4cf4ababc22917538bef553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:42 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:34920
I20260809 16:34:42.012245 2500 tablet_service.cc:1514] Processing DeleteTablet for tablet c85fa45cb4cf4ababc22917538bef553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:42 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60628
I20260809 16:34:42.012490 2966 tablet_service.cc:1514] Processing DeleteTablet for tablet c85fa45cb4cf4ababc22917538bef553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:42 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53018
I20260809 16:34:42.013341 2273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:42.013536 2273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:42.015168 2720 tablet_service.cc:1467] Processing CreateTablet for tablet 896620b1e57c47e4913411aada3d10d0 (DEFAULT_TABLE table=test-table [id=26614671ad7a46aa8bb9bfced5bc4187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:42.021965 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896620b1e57c47e4913411aada3d10d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:42.023450 3116 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Bootstrap starting.
I20260809 16:34:42.023728 2966 tablet_service.cc:1467] Processing CreateTablet for tablet 896620b1e57c47e4913411aada3d10d0 (DEFAULT_TABLE table=test-table [id=26614671ad7a46aa8bb9bfced5bc4187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:42.023850 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896620b1e57c47e4913411aada3d10d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:42.023912 3116 tablet_bootstrap.cc:654] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:42.024636 3116 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: No bootstrap required, opened a new log
I20260809 16:34:42.024691 3116 ts_tablet_manager.cc:1397] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:42.024835 3116 raft_consensus.cc:348] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.024910 3116 raft_consensus.cc:374] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:42.024937 3116 raft_consensus.cc:729] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Initialized, Role: FOLLOWER
I20260809 16:34:42.024981 3116 consensus_queue.cc:260] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.025094 3116 ts_tablet_manager.cc:1428] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:42.025164 3124 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Bootstrap starting.
I20260809 16:34:42.025224 2500 tablet_service.cc:1467] Processing CreateTablet for tablet 896620b1e57c47e4913411aada3d10d0 (DEFAULT_TABLE table=test-table [id=26614671ad7a46aa8bb9bfced5bc4187]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:42.025342 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 896620b1e57c47e4913411aada3d10d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:42.025616 3124 tablet_bootstrap.cc:654] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:42.026293 3124 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: No bootstrap required, opened a new log
I20260809 16:34:42.026350 3124 ts_tablet_manager.cc:1397] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:42.026518 3124 raft_consensus.cc:348] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.026593 3124 raft_consensus.cc:374] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:42.026619 3124 raft_consensus.cc:729] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Initialized, Role: FOLLOWER
I20260809 16:34:42.026666 3124 consensus_queue.cc:260] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.026772 3124 ts_tablet_manager.cc:1428] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:42.026873 3117 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Bootstrap starting.
I20260809 16:34:42.027184 3190 tablet_replica.cc:330] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: stopping tablet replica
I20260809 16:34:42.027319 3117 tablet_bootstrap.cc:654] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:42.027334 3190 raft_consensus.cc:2229] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:42.027393 3190 raft_consensus.cc:2258] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:42.027568 3191 tablet_replica.cc:330] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: stopping tablet replica
I20260809 16:34:42.027678 3191 raft_consensus.cc:2229] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:42.027865 3192 tablet_replica.cc:330] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: stopping tablet replica
I20260809 16:34:42.027966 3192 raft_consensus.cc:2229] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:42.028028 3192 raft_consensus.cc:2258] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:42.028136 3191 raft_consensus.cc:2258] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:42.028357 3117 tablet_bootstrap.cc:492] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: No bootstrap required, opened a new log
I20260809 16:34:42.028422 3117 ts_tablet_manager.cc:1397] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:34:42.028712 3117 raft_consensus.cc:348] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.028800 3117 raft_consensus.cc:374] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:42.028824 3117 raft_consensus.cc:729] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Initialized, Role: FOLLOWER
I20260809 16:34:42.028872 3117 consensus_queue.cc:260] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.028915 3192 ts_tablet_manager.cc:1905] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:42.028971 3117 ts_tablet_manager.cc:1428] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:42.030929 3190 ts_tablet_manager.cc:1905] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:42.032493 3190 ts_tablet_manager.cc:1918] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:42.033298 3191 ts_tablet_manager.cc:1905] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:42.034919 3191 ts_tablet_manager.cc:1918] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:42.035003 3191 log.cc:1196] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/c85fa45cb4cf4ababc22917538bef553
I20260809 16:34:42.035741 3192 ts_tablet_manager.cc:1918] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:42.035825 3192 log.cc:1196] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/c85fa45cb4cf4ababc22917538bef553
I20260809 16:34:42.036090 3192 ts_tablet_manager.cc:1939] T c85fa45cb4cf4ababc22917538bef553 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting consensus metadata
I20260809 16:34:42.036535 2252 catalog_manager.cc:4841] TS 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843): tablet c85fa45cb4cf4ababc22917538bef553 (table test-table [id=3170fbb944344a368dabbeda65d0fc25]) successfully deleted
I20260809 16:34:42.032572 3190 log.cc:1196] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/c85fa45cb4cf4ababc22917538bef553
I20260809 16:34:42.037032 3190 ts_tablet_manager.cc:1939] T c85fa45cb4cf4ababc22917538bef553 P 675f61c968e44d669f5d691361d778f7: Deleting consensus metadata
I20260809 16:34:42.037465 2253 catalog_manager.cc:4841] TS 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809): tablet c85fa45cb4cf4ababc22917538bef553 (table test-table [id=3170fbb944344a368dabbeda65d0fc25]) successfully deleted
I20260809 16:34:42.037568 3191 ts_tablet_manager.cc:1939] T c85fa45cb4cf4ababc22917538bef553 P 906085f716b6467ba8fbb230ee00280a: Deleting consensus metadata
I20260809 16:34:42.037983 2249 catalog_manager.cc:4841] TS 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277): tablet c85fa45cb4cf4ababc22917538bef553 (table test-table [id=3170fbb944344a368dabbeda65d0fc25]) successfully deleted
I20260809 16:34:42.065666 3127 raft_consensus.cc:482] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:42.065796 3127 raft_consensus.cc:504] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.066190 3127 leader_election.cc:288] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:42.070343 3113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.066590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36159 (local address 127.31.15.195:39277)
0809 16:34:42.066637 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:42.066639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:42.066865 (+ 226us) server_negotiation.cc:407] Connection header received
0809 16:34:42.066967 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:42.066969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:42.066991 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:42.067021 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:42.067235 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.067942 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.069433 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.069925 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.069988 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.069999 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:42.070235 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:34:42.070271 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:42.070393 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.066529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36747 (local address 127.31.15.193:33843)
0809 16:34:42.066575 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:42.066577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:42.066764 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:34:42.066870 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:42.066872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:42.066898 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:42.066932 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:42.067169 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.067911 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.069445 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.069989 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.070037 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.070046 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:42.070282 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 16:34:42.070317 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:42.070667 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.066355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.31.15.194:36747)
0809 16:34:42.066686 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:42.066697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:42.066767 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:42.067008 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:42.067009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:42.067011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:42.067089 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.067091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.068020 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.068021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.069326 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.069329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.070109 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.070110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.070193 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.070197 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:42.070521 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 16:34:42.070568 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":42,"thread_start_us":64,"threads_started":1}
I20260809 16:34:42.070942 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896620b1e57c47e4913411aada3d10d0" candidate_uuid: "675f61c968e44d669f5d691361d778f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af" is_pre_election: true
I20260809 16:34:42.071054 2533 raft_consensus.cc:2454] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 675f61c968e44d669f5d691361d778f7 in term 0.
I20260809 16:34:42.071237 2646 leader_election.cc:302] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 675f61c968e44d669f5d691361d778f7; no voters:
I20260809 16:34:42.071404 3127 raft_consensus.cc:2781] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:42.071408 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.066407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39277 (local address 127.31.15.194:36159)
0809 16:34:42.066798 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:42.066809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:42.066893 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:42.067085 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:42.067086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:42.067087 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:42.067163 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.067165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.068053 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.068054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.069324 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.069327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.070082 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.070084 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.070152 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.070155 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:42.071034 (+ 879us) client_negotiation.cc:240] Negotiation successful
0809 16:34:42.071224 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":48,"thread_start_us":54,"threads_started":1}
I20260809 16:34:42.071460 3127 raft_consensus.cc:482] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:42.071748 3127 raft_consensus.cc:3037] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:42.072270 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896620b1e57c47e4913411aada3d10d0" candidate_uuid: "675f61c968e44d669f5d691361d778f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a" is_pre_election: true
I20260809 16:34:42.072360 2996 raft_consensus.cc:2454] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 675f61c968e44d669f5d691361d778f7 in term 0.
I20260809 16:34:42.072715 3127 raft_consensus.cc:504] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.072868 3127 leader_election.cc:288] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:42.073068 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896620b1e57c47e4913411aada3d10d0" candidate_uuid: "675f61c968e44d669f5d691361d778f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af"
I20260809 16:34:42.073176 2533 raft_consensus.cc:3037] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:42.073601 2996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "896620b1e57c47e4913411aada3d10d0" candidate_uuid: "675f61c968e44d669f5d691361d778f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a"
I20260809 16:34:42.073751 2996 raft_consensus.cc:3037] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:42.073974 2533 raft_consensus.cc:2454] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 675f61c968e44d669f5d691361d778f7 in term 1.
I20260809 16:34:42.074147 2646 leader_election.cc:302] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 675f61c968e44d669f5d691361d778f7; no voters:
I20260809 16:34:42.074232 3127 raft_consensus.cc:2781] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:42.074411 3127 raft_consensus.cc:686] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 1 LEADER]: Becoming Leader. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Running, Role: LEADER
I20260809 16:34:42.074487 3127 consensus_queue.cc:237] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:42.075181 2996 raft_consensus.cc:2454] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 675f61c968e44d669f5d691361d778f7 in term 1.
I20260809 16:34:42.075233 2273 catalog_manager.cc:5495] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 675f61c968e44d669f5d691361d778f7 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "675f61c968e44d669f5d691361d778f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } health_report { overall_health: UNKNOWN } } }
I20260809 16:34:42.092777 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.089679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34934 (local address 127.31.15.194:34809)
0809 16:34:42.089720 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:42.089722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:42.089733 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:42.089784 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:42.089785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:42.089809 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:42.089869 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:42.090023 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.090748 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.091801 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.092097 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.092167 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.092378 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:42.092510 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:42.092597 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:42.092662 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:34:42.092689 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:42.098871 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.095816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:41641 (local address 127.31.15.193:33843)
0809 16:34:42.095861 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:42.095863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:42.095871 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:42.095967 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:42.095967 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:42.095977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:42.096040 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:42.096217 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.096908 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.098024 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.098356 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.098414 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.098608 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:42.098640 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:42.098684 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:42.098760 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:34:42.098779 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:42.098920 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.095615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.31.15.194:41641)
0809 16:34:42.095813 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:42.095826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:42.095896 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:42.096055 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:42.096056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:42.096057 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:42.096136 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.096138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.097003 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.097004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.097942 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.097946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.098432 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.098433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.098503 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.098519 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:42.098700 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:42.098702 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:42.098756 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:34:42.098803 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":100,"threads_started":1}
I20260809 16:34:42.101046 2533 raft_consensus.cc:1261] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Refusing update from remote peer 675f61c968e44d669f5d691361d778f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:42.101295 3127 consensus_queue.cc:1035] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:42.101725 2996 raft_consensus.cc:1261] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Refusing update from remote peer 675f61c968e44d669f5d691361d778f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:42.101951 3127 consensus_queue.cc:1035] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:42.110098 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.107610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:55725 (local address 127.31.15.194:34809)
0809 16:34:42.107651 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:42.107652 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:42.107661 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:42.107730 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:42.107731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:42.107742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:42.107795 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:42.107973 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.108622 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.109330 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.109604 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.109682 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.109877 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:42.109912 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:42.109946 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:42.110035 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:34:42.110052 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:34:42.110157 3181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:42.107566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34809 (local address 127.31.15.193:55725)
0809 16:34:42.107619 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:42.107629 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:42.107671 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:42.107841 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:42.107842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:42.107843 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:42.107892 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.107894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.108711 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.108712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.109243 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:42.109244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:42.109726 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:42.109727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:42.109790 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:42.109798 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:42.109969 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:42.109972 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:42.110034 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:34:42.110082 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:34:42.110450 3163 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:34:43.637075 2273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:43.637176 2273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:43.637966 2273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Sending DeleteTablet for 3 replicas of tablet 896620b1e57c47e4913411aada3d10d0
I20260809 16:34:43.638199 2720 tablet_service.cc:1514] Processing DeleteTablet for tablet 896620b1e57c47e4913411aada3d10d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:34920
I20260809 16:34:43.638211 2966 tablet_service.cc:1514] Processing DeleteTablet for tablet 896620b1e57c47e4913411aada3d10d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53018
I20260809 16:34:43.638393 2500 tablet_service.cc:1514] Processing DeleteTablet for tablet 896620b1e57c47e4913411aada3d10d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60628
I20260809 16:34:43.638414 3250 tablet_replica.cc:330] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: stopping tablet replica
I20260809 16:34:43.638502 3250 raft_consensus.cc:2229] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:43.638618 3250 raft_consensus.cc:2258] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.638767 3249 tablet_replica.cc:330] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: stopping tablet replica
I20260809 16:34:43.638834 3249 raft_consensus.cc:2229] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:43.638882 3249 raft_consensus.cc:2258] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.638957 3250 ts_tablet_manager.cc:1905] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.640884 3249 ts_tablet_manager.cc:1905] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.641867 3250 ts_tablet_manager.cc:1918] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:34:43.641927 3250 log.cc:1196] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/896620b1e57c47e4913411aada3d10d0
I20260809 16:34:43.642167 3250 ts_tablet_manager.cc:1939] T 896620b1e57c47e4913411aada3d10d0 P 675f61c968e44d669f5d691361d778f7: Deleting consensus metadata
I20260809 16:34:43.642446 2253 catalog_manager.cc:4841] TS 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809): tablet 896620b1e57c47e4913411aada3d10d0 (table test-table [id=26614671ad7a46aa8bb9bfced5bc4187]) successfully deleted
I20260809 16:34:43.642603 3249 ts_tablet_manager.cc:1918] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:34:43.642656 3249 log.cc:1196] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/896620b1e57c47e4913411aada3d10d0
I20260809 16:34:43.642840 3249 ts_tablet_manager.cc:1939] T 896620b1e57c47e4913411aada3d10d0 P 906085f716b6467ba8fbb230ee00280a: Deleting consensus metadata
I20260809 16:34:43.642983 3251 tablet_replica.cc:330] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: stopping tablet replica
I20260809 16:34:43.643046 3251 raft_consensus.cc:2229] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:43.643086 3251 raft_consensus.cc:2258] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.643224 2249 catalog_manager.cc:4841] TS 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277): tablet 896620b1e57c47e4913411aada3d10d0 (table test-table [id=26614671ad7a46aa8bb9bfced5bc4187]) successfully deleted
I20260809 16:34:43.643373 3251 ts_tablet_manager.cc:1905] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.644575 3251 ts_tablet_manager.cc:1918] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:34:43.644625 3251 log.cc:1196] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/896620b1e57c47e4913411aada3d10d0
I20260809 16:34:43.644785 3251 ts_tablet_manager.cc:1939] T 896620b1e57c47e4913411aada3d10d0 P 1b4222f2b0d242c1ba3a3d10687970af: Deleting consensus metadata
I20260809 16:34:43.645114 2252 catalog_manager.cc:4841] TS 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843): tablet 896620b1e57c47e4913411aada3d10d0 (table test-table [id=26614671ad7a46aa8bb9bfced5bc4187]) successfully deleted
Aug 09 16:34:43 dist-test-slave-21b9 krb5kdc[2204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:34:43 dist-test-slave-21b9 krb5kdc[2204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:43.665310 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:43.656244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.31.15.254:32885)
0809 16:34:43.656371 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:43.656372 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:43.656383 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:43.656453 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:43.656453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:43.656463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:43.656510 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:43.657019 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:43.657558 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:43.658278 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:43.658466 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:43.658508 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:43.663089 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:43.663091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:43.663091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:43.663098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:43.664460 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:43.664752 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:43.664752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:43.664753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:43.664799 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:43.664920 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:43.664920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:43.664920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:43.665018 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:43.665096 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:43.665163 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:34:43.665227 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":79,"threads_started":1}
I20260809 16:34:43.666664 2273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:43.666815 2273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:43.668181 2720 tablet_service.cc:1467] Processing CreateTablet for tablet 9c8f30dcdaf84d6793df77659f35712f (DEFAULT_TABLE table=test-table [id=986068e4549244c69fd1f049ee0bdf3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.668198 2966 tablet_service.cc:1467] Processing CreateTablet for tablet 9c8f30dcdaf84d6793df77659f35712f (DEFAULT_TABLE table=test-table [id=986068e4549244c69fd1f049ee0bdf3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.668210 2500 tablet_service.cc:1467] Processing CreateTablet for tablet 9c8f30dcdaf84d6793df77659f35712f (DEFAULT_TABLE table=test-table [id=986068e4549244c69fd1f049ee0bdf3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.668327 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c8f30dcdaf84d6793df77659f35712f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.668329 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c8f30dcdaf84d6793df77659f35712f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.668330 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c8f30dcdaf84d6793df77659f35712f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.669685 3267 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Bootstrap starting.
I20260809 16:34:43.669704 3264 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Bootstrap starting.
I20260809 16:34:43.669729 3266 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Bootstrap starting.
I20260809 16:34:43.670037 3264 tablet_bootstrap.cc:654] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.670073 3266 tablet_bootstrap.cc:654] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.670099 3267 tablet_bootstrap.cc:654] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.670622 3264 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: No bootstrap required, opened a new log
I20260809 16:34:43.670622 3267 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: No bootstrap required, opened a new log
I20260809 16:34:43.670676 3267 ts_tablet_manager.cc:1397] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.670678 3264 ts_tablet_manager.cc:1397] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.670778 3266 tablet_bootstrap.cc:492] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: No bootstrap required, opened a new log
I20260809 16:34:43.670830 3266 ts_tablet_manager.cc:1397] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.670816 3264 raft_consensus.cc:348] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.670802 3267 raft_consensus.cc:348] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.670874 3264 raft_consensus.cc:374] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.670879 3267 raft_consensus.cc:374] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.670900 3264 raft_consensus.cc:729] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.670900 3267 raft_consensus.cc:729] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.670948 3264 consensus_queue.cc:260] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.670941 3266 raft_consensus.cc:348] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.671005 3266 raft_consensus.cc:374] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.670984 3267 consensus_queue.cc:260] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.671025 3266 raft_consensus.cc:729] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.671087 3266 consensus_queue.cc:260] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.671120 3264 ts_tablet_manager.cc:1428] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:43.671223 3266 ts_tablet_manager.cc:1428] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:43.671543 3267 ts_tablet_manager.cc:1428] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.755470 3281 raft_consensus.cc:482] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:43.755571 3281 raft_consensus.cc:504] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.755733 3281 leader_election.cc:288] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843), 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809)
I20260809 16:34:43.755944 2755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c8f30dcdaf84d6793df77659f35712f" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7" is_pre_election: true
I20260809 16:34:43.755942 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c8f30dcdaf84d6793df77659f35712f" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af" is_pre_election: true
I20260809 16:34:43.756044 2755 raft_consensus.cc:2454] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 0.
I20260809 16:34:43.756049 2533 raft_consensus.cc:2454] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 0.
I20260809 16:34:43.756224 2889 leader_election.cc:302] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 906085f716b6467ba8fbb230ee00280a; no voters:
I20260809 16:34:43.756322 3281 raft_consensus.cc:2781] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:43.756362 3281 raft_consensus.cc:482] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:43.756376 3281 raft_consensus.cc:3037] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:43.757055 3281 raft_consensus.cc:504] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.757165 3281 leader_election.cc:288] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 election: Requested vote from peers 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843), 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809)
I20260809 16:34:43.757346 2755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c8f30dcdaf84d6793df77659f35712f" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7"
I20260809 16:34:43.757355 2533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c8f30dcdaf84d6793df77659f35712f" candidate_uuid: "906085f716b6467ba8fbb230ee00280a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4222f2b0d242c1ba3a3d10687970af"
I20260809 16:34:43.757447 2533 raft_consensus.cc:3037] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:43.757447 2755 raft_consensus.cc:3037] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:43.758258 2755 raft_consensus.cc:2454] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 1.
I20260809 16:34:43.758258 2533 raft_consensus.cc:2454] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906085f716b6467ba8fbb230ee00280a in term 1.
I20260809 16:34:43.758442 2889 leader_election.cc:302] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 906085f716b6467ba8fbb230ee00280a; no voters:
I20260809 16:34:43.758548 3281 raft_consensus.cc:2781] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:43.758704 3281 raft_consensus.cc:686] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 1 LEADER]: Becoming Leader. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Running, Role: LEADER
I20260809 16:34:43.758755 3281 consensus_queue.cc:237] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.759363 2273 catalog_manager.cc:5495] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a reported cstate change: term changed from 0 to 1, leader changed from <none> to 906085f716b6467ba8fbb230ee00280a (127.31.15.195). New cstate: current_term: 1 leader_uuid: "906085f716b6467ba8fbb230ee00280a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } health_report { overall_health: HEALTHY } } }
I20260809 16:34:43.788333 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:43.786118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.31.15.195:39277)
0809 16:34:43.786281 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:43.786283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:43.786289 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:34:43.786308 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:43.786309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:43.786318 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:43.786373 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:43.786504 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:43.786977 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:43.787751 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:43.787950 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:43.788005 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:43.788139 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:43.788167 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:43.788200 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:43.788242 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 16:34:43.788261 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260809 16:34:43.790367 2755 raft_consensus.cc:1261] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Refusing update from remote peer 906085f716b6467ba8fbb230ee00280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:43.790407 2533 raft_consensus.cc:1261] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Refusing update from remote peer 906085f716b6467ba8fbb230ee00280a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:43.790585 3281 consensus_queue.cc:1035] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:43.790704 3285 consensus_queue.cc:1035] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:43.799888 2273 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:43.799970 2273 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:43.800522 2273 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Sending DeleteTablet for 3 replicas of tablet 9c8f30dcdaf84d6793df77659f35712f
I20260809 16:34:43.800736 2720 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c8f30dcdaf84d6793df77659f35712f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:34920
I20260809 16:34:43.800792 2966 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c8f30dcdaf84d6793df77659f35712f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53018
I20260809 16:34:43.800849 3250 tablet_replica.cc:330] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: stopping tablet replica
I20260809 16:34:43.800894 3250 raft_consensus.cc:2229] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:43.800917 3249 tablet_replica.cc:330] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: stopping tablet replica
I20260809 16:34:43.800949 3250 raft_consensus.cc:2258] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.800982 3249 raft_consensus.cc:2229] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:43.801038 3249 raft_consensus.cc:2258] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.800974 2273 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:43.801102 2273 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:43.801091 2500 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c8f30dcdaf84d6793df77659f35712f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:43 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60628
I20260809 16:34:43.801172 3251 tablet_replica.cc:330] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: stopping tablet replica
I20260809 16:34:43.801223 3251 raft_consensus.cc:2229] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:43.801259 3251 raft_consensus.cc:2258] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:43.801330 3249 ts_tablet_manager.cc:1905] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.801589 3251 ts_tablet_manager.cc:1905] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.801591 3250 ts_tablet_manager.cc:1905] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:43.802930 3249 ts_tablet_manager.cc:1918] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:43.802973 3249 log.cc:1196] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/9c8f30dcdaf84d6793df77659f35712f
I20260809 16:34:43.803037 3250 ts_tablet_manager.cc:1918] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:43.803088 3250 log.cc:1196] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/9c8f30dcdaf84d6793df77659f35712f
I20260809 16:34:43.803086 3251 ts_tablet_manager.cc:1918] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:34:43.803118 3249 ts_tablet_manager.cc:1939] T 9c8f30dcdaf84d6793df77659f35712f P 906085f716b6467ba8fbb230ee00280a: Deleting consensus metadata
I20260809 16:34:43.803128 3251 log.cc:1196] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/9c8f30dcdaf84d6793df77659f35712f
I20260809 16:34:43.803313 3250 ts_tablet_manager.cc:1939] T 9c8f30dcdaf84d6793df77659f35712f P 675f61c968e44d669f5d691361d778f7: Deleting consensus metadata
I20260809 16:34:43.803363 2249 catalog_manager.cc:4841] TS 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277): tablet 9c8f30dcdaf84d6793df77659f35712f (table test-table [id=986068e4549244c69fd1f049ee0bdf3a]) successfully deleted
I20260809 16:34:43.803539 3251 ts_tablet_manager.cc:1939] T 9c8f30dcdaf84d6793df77659f35712f P 1b4222f2b0d242c1ba3a3d10687970af: Deleting consensus metadata
I20260809 16:34:43.803791 2252 catalog_manager.cc:4841] TS 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843): tablet 9c8f30dcdaf84d6793df77659f35712f (table test-table [id=986068e4549244c69fd1f049ee0bdf3a]) successfully deleted
I20260809 16:34:43.803823 2253 catalog_manager.cc:4841] TS 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809): tablet 9c8f30dcdaf84d6793df77659f35712f (table test-table [id=986068e4549244c69fd1f049ee0bdf3a]) successfully deleted
I20260809 16:34:43.803872 2720 tablet_service.cc:1467] Processing CreateTablet for tablet ab78ad74983943faa242445cbb2854ff (DEFAULT_TABLE table=test-table [id=354b96106a7d40dea65bc52be4474481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.803872 2500 tablet_service.cc:1467] Processing CreateTablet for tablet ab78ad74983943faa242445cbb2854ff (DEFAULT_TABLE table=test-table [id=354b96106a7d40dea65bc52be4474481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.803903 2966 tablet_service.cc:1467] Processing CreateTablet for tablet ab78ad74983943faa242445cbb2854ff (DEFAULT_TABLE table=test-table [id=354b96106a7d40dea65bc52be4474481]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:43.803969 2720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab78ad74983943faa242445cbb2854ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.803969 2500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab78ad74983943faa242445cbb2854ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.803992 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab78ad74983943faa242445cbb2854ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:43.804721 3264 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Bootstrap starting.
I20260809 16:34:43.805058 3264 tablet_bootstrap.cc:654] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.805063 3266 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Bootstrap starting.
I20260809 16:34:43.805351 3266 tablet_bootstrap.cc:654] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.805563 3267 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af: Bootstrap starting.
I20260809 16:34:43.805612 3264 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: No bootstrap required, opened a new log
I20260809 16:34:43.805648 3264 ts_tablet_manager.cc:1397] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.805758 3264 raft_consensus.cc:348] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.805821 3264 raft_consensus.cc:374] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.805846 3264 raft_consensus.cc:729] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906085f716b6467ba8fbb230ee00280a, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.805891 3267 tablet_bootstrap.cc:654] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:43.805892 3264 consensus_queue.cc:260] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.805989 3264 ts_tablet_manager.cc:1428] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:43.806218 3266 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: No bootstrap required, opened a new log
I20260809 16:34:43.806260 3266 ts_tablet_manager.cc:1397] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:34:43.806384 3267 tablet_bootstrap.cc:492] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af: No bootstrap required, opened a new log
I20260809 16:34:43.806370 3266 raft_consensus.cc:348] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.806416 3266 raft_consensus.cc:374] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.806433 3266 raft_consensus.cc:729] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 675f61c968e44d669f5d691361d778f7, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.806427 3267 ts_tablet_manager.cc:1397] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:43.806471 3266 consensus_queue.cc:260] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.806524 3267 raft_consensus.cc:348] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.806578 3266 ts_tablet_manager.cc:1428] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:43.806596 3267 raft_consensus.cc:374] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:43.806617 3267 raft_consensus.cc:729] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Initialized, Role: FOLLOWER
I20260809 16:34:43.806663 3267 consensus_queue.cc:260] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:43.806746 3267 ts_tablet_manager.cc:1428] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:44.028294 3228 raft_consensus.cc:482] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:44.028401 3228 raft_consensus.cc:504] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:44.028584 3228 leader_election.cc:288] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:44.028776 2755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab78ad74983943faa242445cbb2854ff" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7" is_pre_election: true
I20260809 16:34:44.028810 2995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab78ad74983943faa242445cbb2854ff" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a" is_pre_election: true
I20260809 16:34:44.028883 2755 raft_consensus.cc:2454] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 0.
I20260809 16:34:44.028890 2995 raft_consensus.cc:2454] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 0.
I20260809 16:34:44.029048 2421 leader_election.cc:302] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 675f61c968e44d669f5d691361d778f7; no voters:
I20260809 16:34:44.029182 3228 raft_consensus.cc:2781] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:44.029234 3228 raft_consensus.cc:482] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:44.029261 3228 raft_consensus.cc:3037] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:44.029958 3228 raft_consensus.cc:504] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:44.030078 3228 leader_election.cc:288] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 election: Requested vote from peers 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809), 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277)
I20260809 16:34:44.030222 2755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab78ad74983943faa242445cbb2854ff" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "675f61c968e44d669f5d691361d778f7"
I20260809 16:34:44.030215 2995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab78ad74983943faa242445cbb2854ff" candidate_uuid: "1b4222f2b0d242c1ba3a3d10687970af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "906085f716b6467ba8fbb230ee00280a"
I20260809 16:34:44.030288 2995 raft_consensus.cc:3037] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:44.030292 2755 raft_consensus.cc:3037] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:44.030892 2755 raft_consensus.cc:2454] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 1.
I20260809 16:34:44.030903 2995 raft_consensus.cc:2454] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4222f2b0d242c1ba3a3d10687970af in term 1.
I20260809 16:34:44.031065 2417 leader_election.cc:302] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4222f2b0d242c1ba3a3d10687970af, 906085f716b6467ba8fbb230ee00280a; no voters:
I20260809 16:34:44.031160 3228 raft_consensus.cc:2781] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:44.031210 3228 raft_consensus.cc:686] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [term 1 LEADER]: Becoming Leader. State: Replica: 1b4222f2b0d242c1ba3a3d10687970af, State: Running, Role: LEADER
I20260809 16:34:44.031262 3228 consensus_queue.cc:237] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } }
I20260809 16:34:44.031864 2270 catalog_manager.cc:5495] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193). New cstate: current_term: 1 leader_uuid: "1b4222f2b0d242c1ba3a3d10687970af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4222f2b0d242c1ba3a3d10687970af" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 } health_report { overall_health: UNKNOWN } } }
I20260809 16:34:44.090426 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:44.088211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60634 (local address 127.31.15.193:33843)
0809 16:34:44.088387 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:44.088389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:44.088396 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:44.088417 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:44.088417 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:44.088426 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:44.088481 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:44.088594 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.089083 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.089778 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.090001 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.090066 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:44.090212 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:44.090242 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:44.090280 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:44.090314 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:34:44.090334 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260809 16:34:44.094769 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:44.092807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:42191 (local address 127.31.15.193:33843)
0809 16:34:44.092853 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:44.092854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:44.092860 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:34:44.092923 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:44.092923 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:44.092930 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:44.092969 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:44.093064 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.093500 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.094192 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.094369 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.094455 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:44.094579 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:44.094603 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:44.094640 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:44.094681 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:34:44.094700 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":28}
I20260809 16:34:44.094777 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:44.092653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.31.15.193:42191)
0809 16:34:44.092795 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:44.092806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:44.092860 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:44.092980 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:44.092981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:44.092983 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:34:44.093022 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:44.093023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.093579 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.093580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:44.094113 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:44.094114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:44.094475 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:44.094476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:44.094520 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:44.094524 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:34:44.094645 (+ 121us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:34:44.094646 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:34:44.094671 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 16:34:44.094707 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260809 16:34:44.096657 2995 raft_consensus.cc:1261] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Refusing update from remote peer 1b4222f2b0d242c1ba3a3d10687970af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:44.096660 2755 raft_consensus.cc:1261] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Refusing update from remote peer 1b4222f2b0d242c1ba3a3d10687970af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:44.096869 3215 consensus_queue.cc:1035] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Connected to new peer: Peer: permanent_uuid: "675f61c968e44d669f5d691361d778f7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:44.096967 3228 consensus_queue.cc:1035] T ab78ad74983943faa242445cbb2854ff P 1b4222f2b0d242c1ba3a3d10687970af [LEADER]: Connected to new peer: Peer: permanent_uuid: "906085f716b6467ba8fbb230ee00280a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:34:44.101225 3213 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 16:34:45.619354 2270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:45.619462 2270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:45.620383 2270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abc62ad43ca145c1bf05d0a291dd2c13: Sending DeleteTablet for 3 replicas of tablet ab78ad74983943faa242445cbb2854ff
I20260809 16:34:45.620661 2966 tablet_service.cc:1514] Processing DeleteTablet for tablet ab78ad74983943faa242445cbb2854ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:45 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53018
I20260809 16:34:45.620667 2500 tablet_service.cc:1514] Processing DeleteTablet for tablet ab78ad74983943faa242445cbb2854ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:45 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60628
I20260809 16:34:45.620714 2720 tablet_service.cc:1514] Processing DeleteTablet for tablet ab78ad74983943faa242445cbb2854ff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:45 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:34920
I20260809 16:34:45.620920 3343 tablet_replica.cc:330] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: stopping tablet replica
I20260809 16:34:45.620978 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2395
I20260809 16:34:45.637459 3343 raft_consensus.cc:2229] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:45.637638 3343 raft_consensus.cc:2258] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:45.639132 3343 ts_tablet_manager.cc:1905] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:45.642182 3343 ts_tablet_manager.cc:1918] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:34:45.642376 3343 log.cc:1196] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/ab78ad74983943faa242445cbb2854ff
I20260809 16:34:45.642632 3343 ts_tablet_manager.cc:1939] T ab78ad74983943faa242445cbb2854ff P 675f61c968e44d669f5d691361d778f7: Deleting consensus metadata
I20260809 16:34:45.642982 2253 catalog_manager.cc:4841] TS 675f61c968e44d669f5d691361d778f7 (127.31.15.194:34809): tablet ab78ad74983943faa242445cbb2854ff (table test-table [id=354b96106a7d40dea65bc52be4474481]) successfully deleted
I20260809 16:34:45.658087 3344 tablet_replica.cc:330] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: stopping tablet replica
I20260809 16:34:45.658164 3344 raft_consensus.cc:2229] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:45.658201 3344 raft_consensus.cc:2258] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:45.658470 3344 ts_tablet_manager.cc:1905] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:45.659834 3344 ts_tablet_manager.cc:1918] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 16:34:45.659888 3344 log.cc:1196] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/ab78ad74983943faa242445cbb2854ff
I20260809 16:34:45.660076 3344 ts_tablet_manager.cc:1939] T ab78ad74983943faa242445cbb2854ff P 906085f716b6467ba8fbb230ee00280a: Deleting consensus metadata
I20260809 16:34:45.660440 2249 catalog_manager.cc:4841] TS 906085f716b6467ba8fbb230ee00280a (127.31.15.195:39277): tablet ab78ad74983943faa242445cbb2854ff (table test-table [id=354b96106a7d40dea65bc52be4474481]) successfully deleted
W20260809 16:34:45.663264 2252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:33843 (error 108)
I20260809 16:34:45.663455 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2622
I20260809 16:34:45.668553 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:45.667046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33843 (local address 127.0.0.1:60644)
0809 16:34:45.668446 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:45.668506 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:33843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1376,"thread_start_us":1268,"threads_started":1}
W20260809 16:34:45.668704 2252 catalog_manager.cc:4584] TS 1b4222f2b0d242c1ba3a3d10687970af (127.31.15.193:33843): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab78ad74983943faa242445cbb2854ff: Network error: Client connection negotiation failed: client connection to 127.31.15.193:33843: connect: Connection refused (error 111)
I20260809 16:34:45.674717 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2862
I20260809 16:34:45.686218 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2224
2026-08-09T16:34:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5927 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:45 dist-test-slave-21b9 krb5kdc[3351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:45 dist-test-slave-21b9 krb5kdc[3351](info): set up 2 sockets
Aug 09 16:34:45 dist-test-slave-21b9 krb5kdc[3351](info): commencing operation
krb5kdc: starting...
W20260809 16:34:46.844221 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:46 dist-test-slave-21b9 krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:46.954931 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45953
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:45953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:47.035236 3376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:47.035470 3376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:47.035490 3376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:47.036569 3376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:47.036607 3376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:47.036619 3376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:47.036633 3376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:47.036643 3376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:47.038059 3376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:45953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:47.038265 3376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:47.038442 3376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:47.043354 3384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.043347 3389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.043407 3376 server_base.cc:1034] running on GCE node
W20260809 16:34:47.043347 3385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.043761 3376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:47.044046 3376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:47.045200 3376 hybrid_clock.cc:648] HybridClock initialized: now 1786293287045183 us; error 32 us; skew 500 ppm
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:47.047866 3376 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:47.048302 3376 webserver.cc:460] Webserver started at http://127.31.15.254:39057/ using document root <none> and password file <none>
I20260809 16:34:47.048516 3376 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:47.048560 3376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:47.048661 3376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:47.049736 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "de2faaab7c13411c8a4b8a3b0b471cc5"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "b959ffcd65a89cbc802b951c16dc55ba"
server_key_iv: "c27ebb0cd156ef2a7c0065d182b12ac1"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.050056 3376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "de2faaab7c13411c8a4b8a3b0b471cc5"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "b959ffcd65a89cbc802b951c16dc55ba"
server_key_iv: "c27ebb0cd156ef2a7c0065d182b12ac1"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.051327 3376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:47.052095 3408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.052253 3376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:47.052347 3376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "de2faaab7c13411c8a4b8a3b0b471cc5"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "b959ffcd65a89cbc802b951c16dc55ba"
server_key_iv: "c27ebb0cd156ef2a7c0065d182b12ac1"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.052441 3376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:47.068137 3376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:47.070045 3376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:47.070195 3376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:47.074321 3376 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:45953
I20260809 16:34:47.074362 3496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:45953 every 8 connection(s)
I20260809 16:34:47.074712 3376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:47.075631 3497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:47.077847 3497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: Bootstrap starting.
I20260809 16:34:47.078424 3497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:47.078697 3497 log.cc:824] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:47.079397 3497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: No bootstrap required, opened a new log
I20260809 16:34:47.079401 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3376
I20260809 16:34:47.079481 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:47.079644 31807 external_mini_cluster.cc:1428] Setting key 9373d5e74f82b696aa01bf363cf67f90
I20260809 16:34:47.083061 3497 raft_consensus.cc:348] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } }
I20260809 16:34:47.083230 3497 raft_consensus.cc:374] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:47.083262 3497 raft_consensus.cc:729] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2faaab7c13411c8a4b8a3b0b471cc5, State: Initialized, Role: FOLLOWER
I20260809 16:34:47.083410 3497 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } }
I20260809 16:34:47.083482 3497 raft_consensus.cc:388] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:47.083509 3497 raft_consensus.cc:482] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:47.083551 3497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:47.084188 3497 raft_consensus.cc:504] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } }
I20260809 16:34:47.084288 3497 leader_election.cc:302] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de2faaab7c13411c8a4b8a3b0b471cc5; no voters:
I20260809 16:34:47.084460 3497 leader_election.cc:288] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:47.084498 3506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:47.084671 3506 raft_consensus.cc:686] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [term 1 LEADER]: Becoming Leader. State: Replica: de2faaab7c13411c8a4b8a3b0b471cc5, State: Running, Role: LEADER
I20260809 16:34:47.084724 3497 sys_catalog.cc:564] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:47.084815 3506 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } }
I20260809 16:34:47.085126 3506 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2faaab7c13411c8a4b8a3b0b471cc5. Latest consensus state: current_term: 1 leader_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } } }
I20260809 16:34:47.085198 3506 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:47.085795 3508 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de2faaab7c13411c8a4b8a3b0b471cc5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45953 } } }
I20260809 16:34:47.085929 3508 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5 [sys.catalog]: This master's current role is: LEADER
W20260809 16:34:47.086623 3528 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:34:47.087204 3528 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:34:47.087296 3513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:47.087440 3513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:47.089272 3513 catalog_manager.cc:1347] Generated new cluster ID: f689922986c841e68cf1b2efba059a24
I20260809 16:34:47.089324 3513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:47.091217 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.080135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.31.15.254:45953)
0809 16:34:47.080425 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:47.080435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:47.080466 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:34:47.081230 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:47.081241 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:47.081303 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:47.081460 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:47.082096 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.083951 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.084879 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.085436 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.085551 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.088156 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:47.088160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:47.088161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:47.088173 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:47.089828 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.090148 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.090151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.090151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.090238 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.090411 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.090412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.090413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.090638 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:47.090783 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:47.090885 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 16:34:47.091024 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":183,"threads_started":1}
I20260809 16:34:47.112221 3513 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:47.112744 3513 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:47.116086 3513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de2faaab7c13411c8a4b8a3b0b471cc5: Generated new TSK 0
I20260809 16:34:47.116255 3513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:47.143496 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:47.225841 3536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:47.226042 3536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:47.226066 3536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:47.227150 3536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:47.227190 3536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:47.227217 3536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:47.228686 3536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:47.228885 3536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:47.229076 3536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:47.234302 3546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.234313 3547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.234539 3536 server_base.cc:1034] running on GCE node
W20260809 16:34:47.234313 3550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.234793 3536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:47.235038 3536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:47.236177 3536 hybrid_clock.cc:648] HybridClock initialized: now 1786293287236165 us; error 24 us; skew 500 ppm
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:47.238744 3536 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:47.239187 3536 webserver.cc:460] Webserver started at http://127.31.15.193:40959/ using document root <none> and password file <none>
I20260809 16:34:47.239437 3536 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:47.239487 3536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:47.239643 3536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:47.240638 3536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "53e4cf066f0b4b93afdf396d7ec34274"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "9dacd47822bb439188db6f28ed768fd3"
server_key_iv: "752cde7d896f2b2f6f02ad6118eeb6f4"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.241099 3536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "53e4cf066f0b4b93afdf396d7ec34274"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "9dacd47822bb439188db6f28ed768fd3"
server_key_iv: "752cde7d896f2b2f6f02ad6118eeb6f4"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.242419 3536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:47.243158 3558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.243333 3536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:47.243422 3536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "53e4cf066f0b4b93afdf396d7ec34274"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "9dacd47822bb439188db6f28ed768fd3"
server_key_iv: "752cde7d896f2b2f6f02ad6118eeb6f4"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.243505 3536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:47.256477 3536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:47.258256 3536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:47.258385 3536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:47.258590 3536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:47.258873 3536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:47.258906 3536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.258934 3536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:47.258958 3536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.264348 3536 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:34665
I20260809 16:34:47.264394 3738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:34665 every 8 connection(s)
I20260809 16:34:47.264698 3536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:47.268199 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3536
I20260809 16:34:47.268276 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:47.268428 31807 external_mini_cluster.cc:1428] Setting key b786fe52089169bba2f14502c75ca5f9
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:47.273782 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.265387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:44091 (local address 127.31.15.254:45953)
0809 16:34:47.265431 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:47.265433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:47.265636 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:34:47.266097 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:47.266098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:47.266107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:47.266163 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:47.267032 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.267584 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.268674 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.268901 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.268967 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.271113 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:47.271114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:47.271115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:47.271120 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:47.272353 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.272820 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.272821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.272821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.272881 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.273193 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.273194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.273194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.273298 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:47.273349 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:47.273651 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 16:34:47.273718 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:34:47.274042 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.265353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45953 (local address 127.31.15.193:44091)
0809 16:34:47.265528 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:47.265570 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:47.266023 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:47.266223 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:47.266227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:47.266778 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:47.266963 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.266968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.267669 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.267670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.268593 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.268596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.268986 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.268988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.269122 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.269585 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:47.269592 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:47.271028 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:47.272432 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.272439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.272442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.272751 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.272950 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.272951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.272952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.273123 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.273357 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:47.273363 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:47.273520 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:34:47.273637 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:34:47.273803 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260809 16:34:47.274538 3739 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45953
I20260809 16:34:47.274624 3739 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:47.274802 3739 heartbeater.cc:511] Master 127.31.15.254:45953 requested a full tablet report, sending...
I20260809 16:34:47.275388 3434 ts_manager.cc:194] Registered new tserver with Master: 53e4cf066f0b4b93afdf396d7ec34274 (127.31.15.193:34665)
I20260809 16:34:47.276296 3434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:44091
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:47.293215 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:47.373216 3747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:47.373413 3747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:47.373458 3747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:47.374565 3747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:47.374599 3747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:47.374624 3747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:47.375998 3747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:47.376180 3747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:47.376353 3747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:47.380924 3755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.380923 3758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.380972 3754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.381201 3747 server_base.cc:1034] running on GCE node
I20260809 16:34:47.381382 3747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:47.381594 3747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:47.382720 3747 hybrid_clock.cc:648] HybridClock initialized: now 1786293287382711 us; error 27 us; skew 500 ppm
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:47.385185 3747 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:47.385592 3747 webserver.cc:460] Webserver started at http://127.31.15.194:34717/ using document root <none> and password file <none>
I20260809 16:34:47.385829 3747 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:47.385895 3747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:47.386024 3747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:47.386981 3747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "cc9cf40a558148f4ad56ea72f230e482"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "c322cbbeee340428076d2c2f49bd5771"
server_key_iv: "d6a84d62eb7ff4aebc36b23eb9b69c83"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.387297 3747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "cc9cf40a558148f4ad56ea72f230e482"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "c322cbbeee340428076d2c2f49bd5771"
server_key_iv: "d6a84d62eb7ff4aebc36b23eb9b69c83"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.388492 3747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:47.389142 3768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.389289 3747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:47.389371 3747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "cc9cf40a558148f4ad56ea72f230e482"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "c322cbbeee340428076d2c2f49bd5771"
server_key_iv: "d6a84d62eb7ff4aebc36b23eb9b69c83"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.389421 3747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:47.424340 3747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:47.426484 3747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:47.426602 3747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:47.426798 3747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:47.427076 3747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:47.427107 3747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.427129 3747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:47.427139 3747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.432535 3747 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:35065
I20260809 16:34:47.432664 3935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:35065 every 8 connection(s)
I20260809 16:34:47.432901 3747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:47.438040 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3747
I20260809 16:34:47.438138 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:47.438308 31807 external_mini_cluster.cc:1428] Setting key e908e194c41e2e022d47060563977d5b
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:47.441358 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.433608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36495 (local address 127.31.15.254:45953)
0809 16:34:47.433673 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:47.433675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:47.433845 (+ 170us) server_negotiation.cc:407] Connection header received
0809 16:34:47.434307 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:47.434308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:47.434318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:47.434367 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:47.435281 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.435793 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.436709 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.436886 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.436930 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.438872 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:47.438874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:47.438874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:47.438881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:47.440142 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.440498 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.440499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.440499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.440561 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.440770 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.440771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.440772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.440879 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:47.440933 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:47.441193 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 16:34:47.441270 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:34:47.441519 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.433561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45953 (local address 127.31.15.194:36495)
0809 16:34:47.433728 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:47.433764 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:47.434229 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:47.434449 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:47.434453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:47.434995 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:47.435195 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.435200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.435860 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.435860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.436645 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.436647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.436938 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.436939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.437039 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.437424 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:47.437431 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:47.438786 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:47.440220 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.440227 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.440229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.440433 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.440614 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.440615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.440615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.440716 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.440932 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:47.440938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:47.441074 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 16:34:47.441161 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:34:47.441310 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260809 16:34:47.441928 3936 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45953
I20260809 16:34:47.442021 3936 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:47.442205 3936 heartbeater.cc:511] Master 127.31.15.254:45953 requested a full tablet report, sending...
I20260809 16:34:47.442577 3434 ts_manager.cc:194] Registered new tserver with Master: cc9cf40a558148f4ad56ea72f230e482 (127.31.15.194:35065)
I20260809 16:34:47.443073 3434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:36495
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:47.461606 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:47.543793 3946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:47.544005 3946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:47.544029 3946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:47.545122 3946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:47.545159 3946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:47.545184 3946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:47.546604 3946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:47.546792 3946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:47.546978 3946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:47.551687 3958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.551728 3954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:47.551729 3953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:47.551982 3946 server_base.cc:1034] running on GCE node
I20260809 16:34:47.552170 3946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:47.552359 3946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:47.553499 3946 hybrid_clock.cc:648] HybridClock initialized: now 1786293287553482 us; error 32 us; skew 500 ppm
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:47.555886 3946 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:47.556311 3946 webserver.cc:460] Webserver started at http://127.31.15.195:44715/ using document root <none> and password file <none>
I20260809 16:34:47.556509 3946 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:47.556552 3946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:47.556674 3946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:47.557682 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "5b8de6e8e961410dae6c436177c695ef"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "f2ed09a252e4e897ed5057097f58e56a"
server_key_iv: "f702a5abaa49f54ea1c1777d08cee3e0"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.558018 3946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "5b8de6e8e961410dae6c436177c695ef"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "f2ed09a252e4e897ed5057097f58e56a"
server_key_iv: "f702a5abaa49f54ea1c1777d08cee3e0"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.559226 3946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:47.559921 3968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.560071 3946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:47.560155 3946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "5b8de6e8e961410dae6c436177c695ef"
format_stamp: "Formatted at 2026-08-09 16:34:47 on dist-test-slave-21b9"
server_key: "f2ed09a252e4e897ed5057097f58e56a"
server_key_iv: "f702a5abaa49f54ea1c1777d08cee3e0"
server_key_version: "encryptionkey@0"
I20260809 16:34:47.560221 3946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:47.576476 3946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:47.578452 3946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:47.578650 3946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:47.579044 3946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:47.579449 3946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:47.579487 3946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.579516 3946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:47.579535 3946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:47.586284 3946 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:40061
I20260809 16:34:47.586336 4154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:40061 every 8 connection(s)
I20260809 16:34:47.586658 3946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[3351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293287, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:47.595618 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.587322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:57071 (local address 127.31.15.254:45953)
0809 16:34:47.587363 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:47.587365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:47.587583 (+ 218us) server_negotiation.cc:407] Connection header received
0809 16:34:47.588077 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:47.588079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:47.588093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:47.588143 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:47.589071 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.589590 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.590548 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.590749 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.590804 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.593189 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:47.593191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:47.593191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:47.593200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:47.594310 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.594675 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.594676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.594677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.594741 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:47.594960 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:47.594961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:47.594961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:47.595083 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:47.595145 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:47.595440 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 16:34:47.595526 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:34:47.595780 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:47.587321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45953 (local address 127.31.15.195:57071)
0809 16:34:47.587470 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:47.587505 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:47.587980 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:47.588222 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:47.588226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:47.588781 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:47.588972 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.588977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.589674 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.589676 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.590463 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:47.590466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:47.590817 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:47.590818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:47.590948 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:47.591469 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:47.591479 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:47.593099 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:47.594383 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.594390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.594392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.594607 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.594797 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:47.594797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:47.594798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:47.594904 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:47.595144 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:47.595150 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:47.595309 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 16:34:47.595411 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 16:34:47.595576 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
I20260809 16:34:47.596201 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3946
I20260809 16:34:47.596238 4156 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45953
I20260809 16:34:47.596300 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:47.596329 4156 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:47.596475 31807 external_mini_cluster.cc:1428] Setting key d8c7238878cec2bdc77a7d235572cf40
I20260809 16:34:47.596529 4156 heartbeater.cc:511] Master 127.31.15.254:45953 requested a full tablet report, sending...
I20260809 16:34:47.596925 3434 ts_manager.cc:194] Registered new tserver with Master: 5b8de6e8e961410dae6c436177c695ef (127.31.15.195:40061)
I20260809 16:34:47.597307 3434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:57071
I20260809 16:34:47.597318 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:34:47.609163 3505 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:47.608373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.31.15.254:45953)
0809 16:34:47.608426 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:47.608428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:47.608529 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:34:47.608581 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:47.608582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:47.608590 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:47.608641 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:47.609082 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52674: BlockingRecv error: recv got EOF from 127.0.0.1:52674 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:34:47.610437 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3536
I20260809 16:34:47.623764 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3747
I20260809 16:34:47.642326 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3946
I20260809 16:34:47.648509 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3376
2026-08-09T16:34:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1965 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[4173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[4173](info): set up 2 sockets
Aug 09 16:34:47 dist-test-slave-21b9 krb5kdc[4173](info): commencing operation
krb5kdc: starting...
W20260809 16:34:48.796216 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:48 dist-test-slave-21b9 krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:48.909976 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:43519
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:48.990691 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:48.990887 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:48.990908 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:48.992041 4197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:48.992082 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:48.992095 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:48.992106 4197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:48.992117 4197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:48.993535 4197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:48.993724 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:48.993893 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:48.998363 4215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:48.998435 4210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:48.998395 4197 server_base.cc:1034] running on GCE node
W20260809 16:34:48.998359 4208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:48.998785 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:48.999061 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:49.000211 4197 hybrid_clock.cc:648] HybridClock initialized: now 1786293289000189 us; error 33 us; skew 500 ppm
Aug 09 16:34:49 dist-test-slave-21b9 krb5kdc[4173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293289, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:49.005050 4197 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:49.005450 4197 webserver.cc:460] Webserver started at http://127.31.15.254:39189/ using document root <none> and password file <none>
I20260809 16:34:49.005646 4197 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:49.005693 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:49.005820 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:49.006847 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "0053cad3089f488baa3d43119632ce96"
format_stamp: "Formatted at 2026-08-09 16:34:49 on dist-test-slave-21b9"
server_key: "05c04c14a5397939aac4d04841d3bac2"
server_key_iv: "51c5141427f305f4a24d57ffc8ea4479"
server_key_version: "encryptionkey@0"
I20260809 16:34:49.007181 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "0053cad3089f488baa3d43119632ce96"
format_stamp: "Formatted at 2026-08-09 16:34:49 on dist-test-slave-21b9"
server_key: "05c04c14a5397939aac4d04841d3bac2"
server_key_iv: "51c5141427f305f4a24d57ffc8ea4479"
server_key_version: "encryptionkey@0"
I20260809 16:34:49.008509 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:49.009202 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:49.009351 4197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:49.009435 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "0053cad3089f488baa3d43119632ce96"
format_stamp: "Formatted at 2026-08-09 16:34:49 on dist-test-slave-21b9"
server_key: "05c04c14a5397939aac4d04841d3bac2"
server_key_iv: "51c5141427f305f4a24d57ffc8ea4479"
server_key_version: "encryptionkey@0"
I20260809 16:34:49.009505 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:49.037638 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:49.039562 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:49.039695 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:49.043454 4197 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:43519
I20260809 16:34:49.043510 4300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:43519 every 8 connection(s)
I20260809 16:34:49.044018 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:49.044490 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 4197
I20260809 16:34:49.044574 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:49.044683 4303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:49.044734 31807 external_mini_cluster.cc:1428] Setting key 2fea663e8f13531380eefa626bf990e8
I20260809 16:34:49.047698 4303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96: Bootstrap starting.
I20260809 16:34:49.048419 4303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:49.048739 4303 log.cc:824] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:49.049430 4303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96: No bootstrap required, opened a new log
Aug 09 16:34:49 dist-test-slave-21b9 krb5kdc[4173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:49.051744 4303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } }
I20260809 16:34:49.051919 4303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:49.051975 4303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0053cad3089f488baa3d43119632ce96, State: Initialized, Role: FOLLOWER
I20260809 16:34:49.052110 4303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } }
I20260809 16:34:49.052196 4303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:49.052222 4303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:49.052271 4303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:49.052778 4303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } }
I20260809 16:34:49.052867 4303 leader_election.cc:302] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0053cad3089f488baa3d43119632ce96; no voters:
I20260809 16:34:49.053027 4303 leader_election.cc:288] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:49.053073 4316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:49.053205 4316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [term 1 LEADER]: Becoming Leader. State: Replica: 0053cad3089f488baa3d43119632ce96, State: Running, Role: LEADER
I20260809 16:34:49.053330 4303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:49.053340 4316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } }
I20260809 16:34:49.053736 4318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0053cad3089f488baa3d43119632ce96. Latest consensus state: current_term: 1 leader_uuid: "0053cad3089f488baa3d43119632ce96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } } }
I20260809 16:34:49.053915 4318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:49.054389 4317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0053cad3089f488baa3d43119632ce96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0053cad3089f488baa3d43119632ce96" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 43519 } } }
I20260809 16:34:49.054543 4317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:49.054574 4327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:49.055124 4327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:49.056033 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:49.045208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.31.15.254:43519)
0809 16:34:49.045381 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:49.045389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:49.045414 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:34:49.046006 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:49.046013 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:49.046061 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:49.046172 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:49.046954 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:49.047907 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:49.048938 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:49.049447 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:49.049593 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:49.052201 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:49.052206 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:49.052206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:49.052219 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:49.054256 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:49.054683 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:49.054687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:49.054687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:49.054800 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:49.055078 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:49.055079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:49.055079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:49.055416 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:49.055573 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:49.055672 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:34:49.055815 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260809 16:34:49.057144 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:49.059870 4327 catalog_manager.cc:1347] Generated new cluster ID: a515854746564807a44c41ad7f38af13
I20260809 16:34:49.059926 4327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:49.068284 4327 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:49.068790 4327 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:49.074821 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:49.070714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.31.15.254:43519)
0809 16:34:49.070764 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:49.070766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:49.070931 (+ 165us) server_negotiation.cc:407] Connection header received
0809 16:34:49.070982 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:49.070983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:49.070996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:49.071064 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:49.071473 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:49.072223 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:49.073251 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:49.073571 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:49.073649 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:49.074346 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:49.074347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:49.074348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:49.074574 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:49.074595 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:49.074603 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:49.074671 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:49.074715 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:34:49.074741 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:34:49.078410 4327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0053cad3089f488baa3d43119632ce96: Generated new TSK 0
I20260809 16:34:49.078588 4327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:49.085299 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 4197
2026-08-09T16:34:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1440 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:49 dist-test-slave-21b9 krb5kdc[4359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:49 dist-test-slave-21b9 krb5kdc[4359](info): set up 2 sockets
Aug 09 16:34:49 dist-test-slave-21b9 krb5kdc[4359](info): commencing operation
krb5kdc: starting...
W20260809 16:34:50.237730 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:50.349407 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40053
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:50.428817 4381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:50.428999 4381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:50.429019 4381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:50.430128 4381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:50.430171 4381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:50.430183 4381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:50.430205 4381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:50.430230 4381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:50.431686 4381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:50.431895 4381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:50.432096 4381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:50.436604 4381 server_base.cc:1034] running on GCE node
W20260809 16:34:50.436587 4391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.436587 4389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.436578 4387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:50.436959 4381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:50.437224 4381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:50.438365 4381 hybrid_clock.cc:648] HybridClock initialized: now 1786293290438351 us; error 28 us; skew 500 ppm
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:50.440707 4381 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:50.441133 4381 webserver.cc:460] Webserver started at http://127.31.15.254:39671/ using document root <none> and password file <none>
I20260809 16:34:50.441318 4381 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:50.441365 4381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:50.441496 4381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:50.442497 4381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "b5b7b9c296395d1e26c1c8a9a10dfcd2"
server_key_iv: "6fc5c976f820d9eb24ab0ccc895b4407"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.442826 4381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "b5b7b9c296395d1e26c1c8a9a10dfcd2"
server_key_iv: "6fc5c976f820d9eb24ab0ccc895b4407"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.444105 4381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:50.444741 4401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.444898 4381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:50.444979 4381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "b5b7b9c296395d1e26c1c8a9a10dfcd2"
server_key_iv: "6fc5c976f820d9eb24ab0ccc895b4407"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.445043 4381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:50.453364 4381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:50.455263 4381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:50.455454 4381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:50.459210 4381 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40053
I20260809 16:34:50.459239 4482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40053 every 8 connection(s)
I20260809 16:34:50.459558 4381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:50.460460 4484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:50.462549 4484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa: Bootstrap starting.
I20260809 16:34:50.463065 4484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:50.463361 4484 log.cc:824] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:50.463438 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 4381
I20260809 16:34:50.463554 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:50.463725 31807 external_mini_cluster.cc:1428] Setting key 9f9d93e8bc1377340cebe2838b27d6f8
I20260809 16:34:50.464104 4484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa: No bootstrap required, opened a new log
I20260809 16:34:50.466212 4484 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } }
I20260809 16:34:50.466341 4484 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:50.466372 4484 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab4ce8470cdc447c9e0cae18d9cc2baa, State: Initialized, Role: FOLLOWER
I20260809 16:34:50.466491 4484 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } }
I20260809 16:34:50.466559 4484 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:50.466589 4484 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:50.466634 4484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:50.467133 4484 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } }
I20260809 16:34:50.467226 4484 leader_election.cc:302] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab4ce8470cdc447c9e0cae18d9cc2baa; no voters:
I20260809 16:34:50.467389 4484 leader_election.cc:288] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:50.467453 4493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:50.467636 4484 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:50.467654 4493 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [term 1 LEADER]: Becoming Leader. State: Replica: ab4ce8470cdc447c9e0cae18d9cc2baa, State: Running, Role: LEADER
I20260809 16:34:50.467744 4493 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } }
I20260809 16:34:50.468058 4493 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab4ce8470cdc447c9e0cae18d9cc2baa. Latest consensus state: current_term: 1 leader_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } } }
I20260809 16:34:50.468127 4493 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:50.468189 4494 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab4ce8470cdc447c9e0cae18d9cc2baa" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40053 } } }
I20260809 16:34:50.468264 4494 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:50.468557 4504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:50.469091 4504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:50.470794 4504 catalog_manager.cc:1347] Generated new cluster ID: 013b7888bdfd489295a1f2c784bfdc46
I20260809 16:34:50.470844 4504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:50.475138 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.464190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.31.15.254:40053)
0809 16:34:50.464348 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:50.464356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:50.464391 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:34:50.465296 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:50.465303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:50.465349 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:50.465453 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:50.466181 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.467422 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.468472 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.469056 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.469638 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.471792 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:50.471796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:50.471796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:50.471807 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:50.473565 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.473944 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.473946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.473947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.474047 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.474232 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.474233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.474233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.474488 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:50.474614 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:50.474736 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:34:50.474946 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260809 16:34:50.494077 4504 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:50.494603 4504 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:50.500257 4504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab4ce8470cdc447c9e0cae18d9cc2baa: Generated new TSK 0
I20260809 16:34:50.500414 4504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:50.529800 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:50.608255 4527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:50.608446 4527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:50.608477 4527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:50.609583 4527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:50.609623 4527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:50.609658 4527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:50.611030 4527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:50.611246 4527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:50.611474 4527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:50.615995 4536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.615943 4535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.615943 4539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:50.616175 4527 server_base.cc:1034] running on GCE node
I20260809 16:34:50.616372 4527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:50.616576 4527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:50.617722 4527 hybrid_clock.cc:648] HybridClock initialized: now 1786293290617704 us; error 29 us; skew 500 ppm
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:50.620147 4527 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:50.620574 4527 webserver.cc:460] Webserver started at http://127.31.15.193:42527/ using document root <none> and password file <none>
I20260809 16:34:50.620761 4527 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:50.620807 4527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:50.620932 4527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:50.621863 4527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "93ce77253ec74aa3b6f89f828dc324b4"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "52ee7178143b60989d3423e2ab17ff83"
server_key_iv: "f2cae0c02f7771809c30f4139fce143f"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.622179 4527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "93ce77253ec74aa3b6f89f828dc324b4"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "52ee7178143b60989d3423e2ab17ff83"
server_key_iv: "f2cae0c02f7771809c30f4139fce143f"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.623391 4527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:34:50.624089 4551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.624229 4527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.624310 4527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "93ce77253ec74aa3b6f89f828dc324b4"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "52ee7178143b60989d3423e2ab17ff83"
server_key_iv: "f2cae0c02f7771809c30f4139fce143f"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.624372 4527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:50.634749 4527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:50.636410 4527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:50.636543 4527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:50.636722 4527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:50.636981 4527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:50.637012 4527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.637038 4527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:50.637053 4527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.642298 4527 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:38037
I20260809 16:34:50.642354 4735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:38037 every 8 connection(s)
I20260809 16:34:50.642616 4527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:50.644212 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 4527
I20260809 16:34:50.644316 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:50.644492 31807 external_mini_cluster.cc:1428] Setting key 78c45b523e114ab2b71e09c8813dd5a9
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:50.651612 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.643308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:49603 (local address 127.31.15.254:40053)
0809 16:34:50.643371 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:50.643373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:50.643535 (+ 162us) server_negotiation.cc:407] Connection header received
0809 16:34:50.643955 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:50.643956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:50.643965 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:50.644010 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:50.644958 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.645454 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.646446 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.646659 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.646715 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.648905 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:50.648907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:50.648907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:50.648915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:50.650356 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.650715 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.650716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.650717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.650778 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.650983 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.650984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.650984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.651118 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:50.651190 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:50.651436 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 16:34:50.651520 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:34:50.651684 4739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.643251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40053 (local address 127.31.15.193:49603)
0809 16:34:50.643428 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:50.643462 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:50.643882 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:50.644074 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:50.644077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:50.644665 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:50.644864 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.644869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.645585 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.645586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.646365 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.646367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.646729 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.646730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.646844 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.647341 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:50.647351 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:50.648813 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:50.650431 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.650437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.650438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.650645 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.650841 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.650842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.650842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.650925 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.651175 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:50.651178 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:50.651319 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 16:34:50.651399 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:34:50.651513 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260809 16:34:50.652164 4736 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40053
I20260809 16:34:50.652253 4736 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:50.652411 4736 heartbeater.cc:511] Master 127.31.15.254:40053 requested a full tablet report, sending...
I20260809 16:34:50.652933 4426 ts_manager.cc:194] Registered new tserver with Master: 93ce77253ec74aa3b6f89f828dc324b4 (127.31.15.193:38037)
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
I20260809 16:34:50.653843 4426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:49603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:50.669390 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:50.750581 4742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:50.750799 4742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:50.750829 4742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:50.752038 4742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:50.752089 4742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:50.752117 4742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:50.753582 4742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:50.753809 4742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:50.754026 4742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:50.758922 4753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.758922 4750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.758941 4755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:50.759112 4742 server_base.cc:1034] running on GCE node
I20260809 16:34:50.759343 4742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:50.759619 4742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:50.760761 4742 hybrid_clock.cc:648] HybridClock initialized: now 1786293290760748 us; error 32 us; skew 500 ppm
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:50.763152 4742 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:50.763620 4742 webserver.cc:460] Webserver started at http://127.31.15.194:44069/ using document root <none> and password file <none>
I20260809 16:34:50.763831 4742 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:50.763878 4742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:50.764007 4742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:50.765028 4742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "6fddf1a566b24882a75d1eed62478ef2"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "e10066814ebb00defaeb5fb4cd447685"
server_key_iv: "202423f7204e0f9421b4d743120bee8b"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.765359 4742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "6fddf1a566b24882a75d1eed62478ef2"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "e10066814ebb00defaeb5fb4cd447685"
server_key_iv: "202423f7204e0f9421b4d743120bee8b"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.766582 4742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:50.767215 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.767377 4742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.767458 4742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "6fddf1a566b24882a75d1eed62478ef2"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "e10066814ebb00defaeb5fb4cd447685"
server_key_iv: "202423f7204e0f9421b4d743120bee8b"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.767556 4742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:50.812454 4742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:50.814604 4742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:50.814749 4742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:50.814985 4742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:50.815320 4742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:50.815361 4742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.815393 4742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:50.815411 4742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.821086 4742 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:32823
I20260809 16:34:50.821130 4964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:32823 every 8 connection(s)
I20260809 16:34:50.821439 4742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:50.824653 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 4742
I20260809 16:34:50.824764 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:50.824940 31807 external_mini_cluster.cc:1428] Setting key cb2a4cab64912af4d0c1759ee76e5caf
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:50.830500 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.822173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40147 (local address 127.31.15.254:40053)
0809 16:34:50.822241 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:50.822244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:50.822419 (+ 175us) server_negotiation.cc:407] Connection header received
0809 16:34:50.822881 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:50.822882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:50.822892 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:50.822934 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:50.823868 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.824358 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.825377 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.825673 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.825726 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.827981 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:50.827983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:50.827983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:50.827990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:50.829244 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.829617 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.829618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.829618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.829673 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.829890 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.829891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.829892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.830001 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:50.830057 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:50.830315 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 16:34:50.830401 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:34:50.830657 4969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.822137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40053 (local address 127.31.15.194:40147)
0809 16:34:50.822304 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:50.822341 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:50.822808 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:50.822998 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:50.823002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:50.823603 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:50.823796 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.823801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.824423 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.824424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.825311 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.825314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.825749 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.825750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.825890 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.826357 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:50.826367 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:50.827895 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:50.829320 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.829326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.829329 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.829549 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.829727 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.829728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.829728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.829835 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.830058 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:50.830065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:50.830197 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:34:50.830285 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:34:50.830438 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260809 16:34:50.831067 4965 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40053
I20260809 16:34:50.831157 4965 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:50.831365 4965 heartbeater.cc:511] Master 127.31.15.254:40053 requested a full tablet report, sending...
I20260809 16:34:50.831743 4426 ts_manager.cc:194] Registered new tserver with Master: 6fddf1a566b24882a75d1eed62478ef2 (127.31.15.194:32823)
I20260809 16:34:50.832110 4426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:40147
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:50.850270 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:50.930600 4972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:50.930787 4972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:50.930809 4972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:50.931880 4972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:50.931914 4972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:50.931948 4972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:50.933313 4972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:50.933490 4972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:50.933666 4972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:50.938413 4986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.938477 4980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:50.938433 4981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:50.938692 4972 server_base.cc:1034] running on GCE node
I20260809 16:34:50.938884 4972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:50.939100 4972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:50.940266 4972 hybrid_clock.cc:648] HybridClock initialized: now 1786293290940246 us; error 29 us; skew 500 ppm
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:50.942957 4972 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:50.943418 4972 webserver.cc:460] Webserver started at http://127.31.15.195:36593/ using document root <none> and password file <none>
I20260809 16:34:50.943653 4972 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:50.943706 4972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:50.943841 4972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:50.944926 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "f7249a7a450440eeb54c5433b3485598"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "bd4ad06fbd7205b6f35e1c7ad1e35e28"
server_key_iv: "11bd5185201178fa7ff75aebfe57e454"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.945274 4972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "f7249a7a450440eeb54c5433b3485598"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "bd4ad06fbd7205b6f35e1c7ad1e35e28"
server_key_iv: "11bd5185201178fa7ff75aebfe57e454"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.946575 4972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:50.947365 4996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.947556 4972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:50.947656 4972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "f7249a7a450440eeb54c5433b3485598"
format_stamp: "Formatted at 2026-08-09 16:34:50 on dist-test-slave-21b9"
server_key: "bd4ad06fbd7205b6f35e1c7ad1e35e28"
server_key_iv: "11bd5185201178fa7ff75aebfe57e454"
server_key_version: "encryptionkey@0"
I20260809 16:34:50.947733 4972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:50.955986 4972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:50.957788 4972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:50.957921 4972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:50.958143 4972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:50.958457 4972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:50.958496 4972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.958520 4972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:50.958549 4972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:50.964680 4972 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:42341
I20260809 16:34:50.964718 5186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:42341 every 8 connection(s)
I20260809 16:34:50.965025 4972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:50.965533 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 4972
I20260809 16:34:50.965659 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:50.965884 31807 external_mini_cluster.cc:1428] Setting key 9760fa4597582f9cd9743650fbc97402
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:50.973905 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.965719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:43261 (local address 127.31.15.254:40053)
0809 16:34:50.965769 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:50.965772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:50.966007 (+ 235us) server_negotiation.cc:407] Connection header received
0809 16:34:50.966581 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:50.966583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:50.966597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:50.966666 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:50.967674 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.968265 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.969309 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.969503 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.969555 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.971606 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:50.971607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:50.971608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:50.971615 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:50.972660 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.973018 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.973019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.973019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.973072 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.973282 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.973283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.973284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.973390 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:50.973447 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:50.973737 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 16:34:50.973818 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:34:50.974056 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.965719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40053 (local address 127.31.15.195:43261)
0809 16:34:50.965887 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:50.965923 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:50.966486 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:50.966719 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:50.966723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:50.967403 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:50.967591 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.967595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.968343 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.968344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.969240 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:50.969243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.969571 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.969571 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:50.969671 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.970121 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:50.970129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:50.971521 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:50.972732 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.972738 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.972740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.972943 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.973124 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:50.973125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:50.973125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:50.973226 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:50.973444 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:50.973451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:50.973606 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 16:34:50.973701 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:34:50.973848 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260809 16:34:50.974490 5189 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40053
I20260809 16:34:50.974584 5189 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:50.974763 5189 heartbeater.cc:511] Master 127.31.15.254:40053 requested a full tablet report, sending...
I20260809 16:34:50.975140 4426 ts_manager.cc:194] Registered new tserver with Master: f7249a7a450440eeb54c5433b3485598 (127.31.15.195:42341)
I20260809 16:34:50.975549 4426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:43261
I20260809 16:34:50.977542 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 16:34:50 dist-test-slave-21b9 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293290, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:50.999454 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:50.993200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.31.15.254:40053)
0809 16:34:50.993255 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:50.993257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:50.993348 (+ 91us) server_negotiation.cc:407] Connection header received
0809 16:34:50.993418 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:50.993419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:50.993429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:50.993468 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:50.994023 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.994491 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.995221 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:50.995437 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:50.995488 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:50.997509 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:50.997510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:50.997511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:50.997517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:50.998548 (+ 1031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.998823 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.998824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.998824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.998866 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:50.998999 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:50.999000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:50.999000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:50.999129 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:50.999198 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:50.999322 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:34:50.999388 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 16:34:50.999660 4426 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60854
I20260809 16:34:51.000026 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 4527
I20260809 16:34:51.011687 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 4742
I20260809 16:34:51.018123 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 4972
I20260809 16:34:51.032958 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 4381
2026-08-09T16:34:51Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1940 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:51 dist-test-slave-21b9 krb5kdc[5213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:51 dist-test-slave-21b9 krb5kdc[5213](info): set up 2 sockets
Aug 09 16:34:51 dist-test-slave-21b9 krb5kdc[5213](info): commencing operation
krb5kdc: starting...
W20260809 16:34:52.185343 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:52.297767 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:37359
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:37359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:52.379866 5237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:52.380066 5237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:52.380087 5237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:52.381204 5237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:52.381244 5237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:52.381258 5237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:52.381279 5237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:52.381302 5237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:52.382871 5237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:37359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:37359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:52.383102 5237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:52.383335 5237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:52.387768 5245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:52.387982 5237 server_base.cc:1034] running on GCE node
W20260809 16:34:52.387787 5248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.387771 5244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:52.388276 5237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:52.388573 5237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:52.389721 5237 hybrid_clock.cc:648] HybridClock initialized: now 1786293292389707 us; error 27 us; skew 500 ppm
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:52.392078 5237 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:52.392524 5237 webserver.cc:460] Webserver started at http://127.31.15.254:42647/ using document root <none> and password file <none>
I20260809 16:34:52.392745 5237 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:52.392791 5237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:52.392916 5237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:52.393913 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "0bbeaee4ee1c4508a668c59008def7b3"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "d543bbfd5ae60b912c483463f69582e1"
server_key_iv: "879ede7f8b0dca2838832a4db7be3597"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.394282 5237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "0bbeaee4ee1c4508a668c59008def7b3"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "d543bbfd5ae60b912c483463f69582e1"
server_key_iv: "879ede7f8b0dca2838832a4db7be3597"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.395537 5237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:34:52.396220 5258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.396358 5237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:52.396478 5237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "0bbeaee4ee1c4508a668c59008def7b3"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "d543bbfd5ae60b912c483463f69582e1"
server_key_iv: "879ede7f8b0dca2838832a4db7be3597"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.396540 5237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:52.413902 5237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:52.415752 5237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:52.415885 5237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:52.419701 5237 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:37359
I20260809 16:34:52.419739 5356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:37359 every 8 connection(s)
I20260809 16:34:52.420200 5237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:52.420849 5361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:52.422514 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 5237
I20260809 16:34:52.422617 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:52.422783 31807 external_mini_cluster.cc:1428] Setting key ff6991d770cc21bb06621e49dcbfa8cb
I20260809 16:34:52.423184 5361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3: Bootstrap starting.
I20260809 16:34:52.423877 5361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:52.424206 5361 log.cc:824] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:52.425101 5361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3: No bootstrap required, opened a new log
I20260809 16:34:52.427235 5361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } }
I20260809 16:34:52.427449 5361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:52.427488 5361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbeaee4ee1c4508a668c59008def7b3, State: Initialized, Role: FOLLOWER
I20260809 16:34:52.427618 5361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } }
I20260809 16:34:52.427703 5361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:52.427771 5361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:52.427829 5361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:52.428331 5361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } }
I20260809 16:34:52.428417 5361 leader_election.cc:302] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bbeaee4ee1c4508a668c59008def7b3; no voters:
I20260809 16:34:52.428562 5361 leader_election.cc:288] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:52.428606 5371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:52.428725 5371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [term 1 LEADER]: Becoming Leader. State: Replica: 0bbeaee4ee1c4508a668c59008def7b3, State: Running, Role: LEADER
I20260809 16:34:52.428838 5361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:52.428871 5371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } }
I20260809 16:34:52.429181 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bbeaee4ee1c4508a668c59008def7b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } } }
I20260809 16:34:52.429260 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:52.429167 5371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bbeaee4ee1c4508a668c59008def7b3. Latest consensus state: current_term: 1 leader_uuid: "0bbeaee4ee1c4508a668c59008def7b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bbeaee4ee1c4508a668c59008def7b3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 37359 } } }
I20260809 16:34:52.429589 5371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:52.429961 5380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:52.430591 5380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:52.432399 5380 catalog_manager.cc:1347] Generated new cluster ID: 2c097636979c4347927ae564ab96e967
I20260809 16:34:52.432480 5380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:52.433485 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.423262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.31.15.254:37359)
0809 16:34:52.423425 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.423433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.423465 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:34:52.423885 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.423892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.423936 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.424066 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.424784 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.425727 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.426748 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.427174 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.427255 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.429847 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.429851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.429851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.429863 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.431814 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.432194 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.432197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.432198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.432298 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.432523 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.432524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.432525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.432837 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.432990 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.433135 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.433275 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260809 16:34:52.446612 5380 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:52.447508 5380 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:52.451320 5380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bbeaee4ee1c4508a668c59008def7b3: Generated new TSK 0
I20260809 16:34:52.451499 5380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:52.477346 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:52.558889 5403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:52.559087 5403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:52.559116 5403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:52.560267 5403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:52.560309 5403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:52.560343 5403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:52.561818 5403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:52.562059 5403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:52.562271 5403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:52.566922 5413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.566946 5417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.566918 5414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:52.567175 5403 server_base.cc:1034] running on GCE node
I20260809 16:34:52.567440 5403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:52.567656 5403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:52.568812 5403 hybrid_clock.cc:648] HybridClock initialized: now 1786293292568801 us; error 28 us; skew 500 ppm
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:52.571348 5403 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:52.571763 5403 webserver.cc:460] Webserver started at http://127.31.15.193:37117/ using document root <none> and password file <none>
I20260809 16:34:52.571988 5403 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:52.572036 5403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:52.572175 5403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:52.573175 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "f2c011f777bd47d582d77a91e51415a6"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "ebad1e290c5ed8049f69af5363356432"
server_key_iv: "5b148921384c086269d7255691132157"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.573491 5403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "f2c011f777bd47d582d77a91e51415a6"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "ebad1e290c5ed8049f69af5363356432"
server_key_iv: "5b148921384c086269d7255691132157"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.574719 5403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:52.575416 5431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.575584 5403 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:52.575667 5403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "f2c011f777bd47d582d77a91e51415a6"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "ebad1e290c5ed8049f69af5363356432"
server_key_iv: "5b148921384c086269d7255691132157"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.575732 5403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:52.591500 5403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:52.593547 5403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:52.593681 5403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:52.593876 5403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:52.594169 5403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:52.594202 5403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.594229 5403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:52.594322 5403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.599876 5403 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:33721
I20260809 16:34:52.599985 5598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:33721 every 8 connection(s)
I20260809 16:34:52.600263 5403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:52.603142 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 5403
I20260809 16:34:52.603253 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:52.603462 31807 external_mini_cluster.cc:1428] Setting key c18734032674f22eb5438579491f4e18
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:52.609508 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.600920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:37553 (local address 127.31.15.254:37359)
0809 16:34:52.600981 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.600983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.601177 (+ 194us) server_negotiation.cc:407] Connection header received
0809 16:34:52.602070 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.602071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.602080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.602116 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.602655 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.603181 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.604410 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.604637 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.604695 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.606663 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.606664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.606665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.606671 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.607843 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.608206 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.608207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.608208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.608261 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.608475 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.608476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.608477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.608713 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.608790 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.609197 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.609354 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:34:52.609551 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.600904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:37359 (local address 127.31.15.193:37553)
0809 16:34:52.601057 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:52.601100 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:52.601970 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:52.602177 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:52.602180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:52.602412 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:52.602594 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.602598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.603299 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.603300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.604328 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.604331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.604713 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.604714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.604846 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.605291 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:52.605300 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:52.606576 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:52.607915 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.607922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.607925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.608139 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.608317 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.608317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.608318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.608418 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.608809 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:52.608817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:52.609013 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 16:34:52.609152 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 16:34:52.609323 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260809 16:34:52.610001 5602 heartbeater.cc:347] Connected to a master server at 127.31.15.254:37359
I20260809 16:34:52.610100 5602 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:52.610273 5602 heartbeater.cc:511] Master 127.31.15.254:37359 requested a full tablet report, sending...
I20260809 16:34:52.610792 5292 ts_manager.cc:194] Registered new tserver with Master: f2c011f777bd47d582d77a91e51415a6 (127.31.15.193:33721)
I20260809 16:34:52.611440 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:37553
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:52.628299 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:52.707765 5611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:52.707948 5611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:52.707970 5611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:52.709088 5611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:52.709146 5611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:52.709174 5611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:52.710632 5611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:52.710837 5611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:52.711045 5611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:34:52.715803 5611 server_base.cc:1034] running on GCE node
W20260809 16:34:52.715734 5620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.715734 5626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.715742 5624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:52.716190 5611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:52.716423 5611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:52.717563 5611 hybrid_clock.cc:648] HybridClock initialized: now 1786293292717544 us; error 32 us; skew 500 ppm
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:52.719974 5611 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:52.720347 5611 webserver.cc:460] Webserver started at http://127.31.15.194:34787/ using document root <none> and password file <none>
I20260809 16:34:52.720518 5611 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:52.720556 5611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:52.720647 5611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:52.721587 5611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "a085ca0dd49f4c4a9fe040de0d8420f1"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "19ece9e88520d79dee170e39edaee01c"
server_key_iv: "31939d578732437f12f80b3017d768a2"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.721885 5611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "a085ca0dd49f4c4a9fe040de0d8420f1"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "19ece9e88520d79dee170e39edaee01c"
server_key_iv: "31939d578732437f12f80b3017d768a2"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.723109 5611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:52.723817 5635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.723958 5611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.724048 5611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "a085ca0dd49f4c4a9fe040de0d8420f1"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "19ece9e88520d79dee170e39edaee01c"
server_key_iv: "31939d578732437f12f80b3017d768a2"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.724102 5611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:52.752240 5611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:52.754465 5611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:52.754583 5611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:52.754781 5611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:52.755105 5611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:52.755141 5611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.755165 5611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:52.755189 5611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.760579 5611 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:38717
I20260809 16:34:52.760637 5812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:38717 every 8 connection(s)
I20260809 16:34:52.760926 5611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:52.763235 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 5611
I20260809 16:34:52.763372 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:52.763589 31807 external_mini_cluster.cc:1428] Setting key 33c6c3c2af0afdb7c43d2413c784ca36
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:52.770291 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.761643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:45943 (local address 127.31.15.254:37359)
0809 16:34:52.761699 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.761701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.761868 (+ 167us) server_negotiation.cc:407] Connection header received
0809 16:34:52.762766 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.762767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.762778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.762850 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.763489 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.764148 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.765145 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.765397 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.765480 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.767626 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.767627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.767628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.767634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.768881 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.769386 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.769387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.769387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.769434 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.769701 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.769701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.769702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.769796 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.769843 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.770169 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.770227 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":31}
I20260809 16:34:52.770560 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.761600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:37359 (local address 127.31.15.194:45943)
0809 16:34:52.761747 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:52.761788 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:52.762675 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:52.762906 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:52.762909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:52.763149 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:52.763396 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.763403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.764245 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.764246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.765052 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.765054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.765475 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.765476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.765568 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.766043 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:52.766051 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:52.767526 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:52.768965 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.768974 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.768976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.769300 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.769493 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.769494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.769494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.769636 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.769854 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:52.769859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:52.770032 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:34:52.770161 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 16:34:52.770328 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260809 16:34:52.771121 5813 heartbeater.cc:347] Connected to a master server at 127.31.15.254:37359
I20260809 16:34:52.771229 5813 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:52.771483 5813 heartbeater.cc:511] Master 127.31.15.254:37359 requested a full tablet report, sending...
I20260809 16:34:52.771917 5292 ts_manager.cc:194] Registered new tserver with Master: a085ca0dd49f4c4a9fe040de0d8420f1 (127.31.15.194:38717)
I20260809 16:34:52.772264 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:45943
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:52.789698 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:36521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:52.870918 5822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:52.871127 5822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:52.871161 5822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:52.872310 5822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:52.872359 5822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:52.872407 5822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:52.873899 5822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:36521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:37359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:52.874130 5822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:52.874343 5822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:52.879163 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.879146 5834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:52.879146 5830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:52.879402 5822 server_base.cc:1034] running on GCE node
I20260809 16:34:52.879618 5822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:52.879834 5822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:52.880970 5822 hybrid_clock.cc:648] HybridClock initialized: now 1786293292880953 us; error 35 us; skew 500 ppm
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:52.883423 5822 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:52.883880 5822 webserver.cc:460] Webserver started at http://127.31.15.195:36543/ using document root <none> and password file <none>
I20260809 16:34:52.884088 5822 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:52.884136 5822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:52.884267 5822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:52.885286 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "8e6095cbb17e480aa0303066358acb8f"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "112090780fd2c7aac8198590760e6573"
server_key_iv: "95c376a00dc169c4523f96158515057e"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.885622 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "8e6095cbb17e480aa0303066358acb8f"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "112090780fd2c7aac8198590760e6573"
server_key_iv: "95c376a00dc169c4523f96158515057e"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.886988 5822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:52.887750 5847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.887909 5822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.887996 5822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "8e6095cbb17e480aa0303066358acb8f"
format_stamp: "Formatted at 2026-08-09 16:34:52 on dist-test-slave-21b9"
server_key: "112090780fd2c7aac8198590760e6573"
server_key_iv: "95c376a00dc169c4523f96158515057e"
server_key_version: "encryptionkey@0"
I20260809 16:34:52.888062 5822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:52.924602 5822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:52.926599 5822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:52.926749 5822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:52.926982 5822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:52.927299 5822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:52.927340 5822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.927371 5822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:52.927395 5822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:52.933125 5822 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:34929
I20260809 16:34:52.933177 6018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:34929 every 8 connection(s)
I20260809 16:34:52.933550 5822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:52.934406 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 5822
I20260809 16:34:52.934487 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:52.934679 31807 external_mini_cluster.cc:1428] Setting key 3b0aba5225f8ed80e233afba5c244f59
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:52.942348 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.934231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:58729 (local address 127.31.15.254:37359)
0809 16:34:52.934272 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.934275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.934494 (+ 219us) server_negotiation.cc:407] Connection header received
0809 16:34:52.935664 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.935665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.935677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.935732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.936320 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.936817 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.937768 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.937976 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.938031 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.939996 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.939998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.939998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.940005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.941051 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.941397 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.941399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.941399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.941470 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.941766 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.941767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.941768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.941880 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.941945 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.942179 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.942260 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:34:52.942512 6022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.934209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:37359 (local address 127.31.15.195:58729)
0809 16:34:52.934360 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:52.934401 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:52.935575 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:52.935802 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:52.935806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:52.936053 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:52.936235 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.936239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.936897 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.936898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.937689 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:52.937691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.938044 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.938045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:52.938172 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.938571 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:52.938578 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:52.939911 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:52.941123 (+ 1212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.941130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.941132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.941333 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.941528 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:52.941529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:52.941529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:52.941703 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:52.941938 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:52.941944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:52.942064 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 16:34:52.942151 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:34:52.942302 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260809 16:34:52.942881 6019 heartbeater.cc:347] Connected to a master server at 127.31.15.254:37359
I20260809 16:34:52.942967 6019 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:52.943141 6019 heartbeater.cc:511] Master 127.31.15.254:37359 requested a full tablet report, sending...
I20260809 16:34:52.943514 5292 ts_manager.cc:194] Registered new tserver with Master: 8e6095cbb17e480aa0303066358acb8f (127.31.15.195:34929)
I20260809 16:34:52.943866 5292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:58729
I20260809 16:34:52.946211 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:52.968940 6028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.961893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.31.15.193:33721)
0809 16:34:52.962053 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.962057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.962066 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:52.962091 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.962092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.962105 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.962171 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.962692 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.963422 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.964169 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.964458 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.964503 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.966499 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.966507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.966508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.966514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.967827 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.968139 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.968142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.968142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.968191 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.968428 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.968429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.968430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.968589 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.968696 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.968781 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.968844 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
Aug 09 16:34:52 dist-test-slave-21b9 krb5kdc[5213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:52.976236 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:52.970181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34362 (local address 127.31.15.254:37359)
0809 16:34:52.970227 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:52.970229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:52.970240 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:52.970295 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:52.970296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:52.970304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:52.970350 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:52.970775 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.971260 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.971878 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:52.972058 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:52.972101 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:52.974078 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:52.974079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:52.974080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:52.974086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:52.975249 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.975627 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.975628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.975628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.975685 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:52.975826 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:52.975826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:52.975826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:52.975963 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:52.976044 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:52.976111 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:34:52.976174 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 16:34:52.976456 5292 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34362
I20260809 16:34:52.976778 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 5403
I20260809 16:34:52.992237 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 5611
I20260809 16:34:53.014495 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 5822
I20260809 16:34:53.023335 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 5237
2026-08-09T16:34:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1990 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:53 dist-test-slave-21b9 krb5kdc[6034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:53 dist-test-slave-21b9 krb5kdc[6034](info): set up 2 sockets
Aug 09 16:34:53 dist-test-slave-21b9 krb5kdc[6034](info): commencing operation
krb5kdc: starting...
W20260809 16:34:54.177455 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:54.291069 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:42721
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:54.370519 6053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:54.370726 6053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:54.370759 6053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:54.371898 6053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:54.371937 6053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:54.371956 6053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:54.371973 6053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:54.371987 6053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:54.373463 6053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:42721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:54.373699 6053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:54.373891 6053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:54.378355 6064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.378414 6063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.378350 6066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:54.378381 6053 server_base.cc:1034] running on GCE node
I20260809 16:34:54.378746 6053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:54.379019 6053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:54.380164 6053 hybrid_clock.cc:648] HybridClock initialized: now 1786293294380150 us; error 33 us; skew 500 ppm
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:54.385501 6053 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:54.385823 6053 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:54.385875 6053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:54.386013 6053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:54.387017 6053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "27cd07e2064f46578280d0c1f158f97b"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "974f4ba0845d53b3a8d7210f1816d9c6"
server_key_iv: "b5f9e6613eff8d811f7d3379a8588670"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.387400 6053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "27cd07e2064f46578280d0c1f158f97b"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "974f4ba0845d53b3a8d7210f1816d9c6"
server_key_iv: "b5f9e6613eff8d811f7d3379a8588670"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.388644 6053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:34:54.389339 6074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.389469 6053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:54.389555 6053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "27cd07e2064f46578280d0c1f158f97b"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "974f4ba0845d53b3a8d7210f1816d9c6"
server_key_iv: "b5f9e6613eff8d811f7d3379a8588670"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.389616 6053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:54.420364 6053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:54.422540 6053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:54.422685 6053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:54.426486 6053 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:42721
I20260809 16:34:54.426517 6153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:42721 every 8 connection(s)
I20260809 16:34:54.426782 6053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:54.427834 6158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:54.430063 6158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: Bootstrap starting.
I20260809 16:34:54.430606 6158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:54.430843 6158 log.cc:824] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:54.431506 6158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: No bootstrap required, opened a new log
I20260809 16:34:54.433032 6158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } }
I20260809 16:34:54.433166 6158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:54.433187 6158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27cd07e2064f46578280d0c1f158f97b, State: Initialized, Role: FOLLOWER
I20260809 16:34:54.433293 6158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } }
I20260809 16:34:54.433368 6158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:54.433396 6158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:54.433426 6158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:54.434145 6158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } }
I20260809 16:34:54.434230 6158 leader_election.cc:302] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27cd07e2064f46578280d0c1f158f97b; no voters:
I20260809 16:34:54.434346 6158 leader_election.cc:288] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:54.434406 6162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:54.434573 6158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:54.434600 6162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [term 1 LEADER]: Becoming Leader. State: Replica: 27cd07e2064f46578280d0c1f158f97b, State: Running, Role: LEADER
I20260809 16:34:54.434754 6162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } }
I20260809 16:34:54.435099 6164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27cd07e2064f46578280d0c1f158f97b. Latest consensus state: current_term: 1 leader_uuid: "27cd07e2064f46578280d0c1f158f97b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } } }
I20260809 16:34:54.435101 6163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27cd07e2064f46578280d0c1f158f97b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cd07e2064f46578280d0c1f158f97b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42721 } } }
I20260809 16:34:54.435292 6164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:54.435299 6163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:54.435665 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 6053
I20260809 16:34:54.435698 6175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:54.435778 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:54.435956 31807 external_mini_cluster.cc:1428] Setting key bd65618aae77799982fd0b25323cf3ec
I20260809 16:34:54.436172 6175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:54.437741 6175 catalog_manager.cc:1347] Generated new cluster ID: 4934e5c578a14e7c90d212e0402554d5
I20260809 16:34:54.437788 6175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:54.445044 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.436544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.31.15.254:42721)
0809 16:34:54.436748 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.436758 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.436860 (+ 102us) server_negotiation.cc:407] Connection header received
0809 16:34:54.437003 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.437009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.437052 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.437164 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:54.437820 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.438731 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.439517 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.439955 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.440048 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.442102 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:54.442105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:54.442106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:54.442117 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:54.443687 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.444014 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.444016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.444017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.444096 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.444264 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.444265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.444265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.444489 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:54.444636 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.444725 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.444856 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":158,"thread_start_us":60,"threads_started":1}
I20260809 16:34:54.447651 6175 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:54.448168 6175 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:54.454600 6175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: Generated new TSK 0
I20260809 16:34:54.454761 6175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:54.484309 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:54.578781 6193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:54.578985 6193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:54.579010 6193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:54.580212 6193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:54.580263 6193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:54.580295 6193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:54.581837 6193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:54.582087 6193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:54.582314 6193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:54.587119 6199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.587147 6200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.587131 6202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:54.587321 6193 server_base.cc:1034] running on GCE node
I20260809 16:34:54.587587 6193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:54.587803 6193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:54.588953 6193 hybrid_clock.cc:648] HybridClock initialized: now 1786293294588934 us; error 28 us; skew 500 ppm
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:54.591387 6193 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:54.591781 6193 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:54.591836 6193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:54.591966 6193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:54.592976 6193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "bc59b533ccf0430cb10db3c0a941ef1a"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "841eb522252441651b1bea954b41eee3"
server_key_iv: "d3856b607b1b252467a24c4578d62a21"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.593300 6193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "bc59b533ccf0430cb10db3c0a941ef1a"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "841eb522252441651b1bea954b41eee3"
server_key_iv: "d3856b607b1b252467a24c4578d62a21"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.594477 6193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:54.595095 6211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.595239 6193 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:54.595350 6193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "bc59b533ccf0430cb10db3c0a941ef1a"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "841eb522252441651b1bea954b41eee3"
server_key_iv: "d3856b607b1b252467a24c4578d62a21"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.595436 6193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:54.606642 6193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:54.608255 6193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:54.608379 6193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:54.608573 6193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:54.608847 6193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:54.608880 6193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.608907 6193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:54.608924 6193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.614439 6193 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:46253
I20260809 16:34:54.614475 6403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:46253 every 8 connection(s)
I20260809 16:34:54.614708 6193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:54.619251 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 6193
I20260809 16:34:54.619398 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:54.619598 31807 external_mini_cluster.cc:1428] Setting key ae349f080f0e6b4f3131c0bf616bc4c9
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:54.623461 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.615355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:38821 (local address 127.31.15.254:42721)
0809 16:34:54.615447 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.615450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.615598 (+ 148us) server_negotiation.cc:407] Connection header received
0809 16:34:54.616058 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.616059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.616068 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.616117 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:54.616970 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.617488 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.618416 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.618619 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.618668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.620908 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:54.620910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:54.620910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:54.620921 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:54.622187 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.622562 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.622563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.622564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.622619 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.622841 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.622842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.622842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.622952 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:54.623014 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.623265 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.623372 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:34:54.623636 6406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.615332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:42721 (local address 127.31.15.193:38821)
0809 16:34:54.615490 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.615532 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.615982 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.616178 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.616181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.616424 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:54.616882 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.616888 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.617554 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.617554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.618329 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.618331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.618675 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.618676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.618770 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.619187 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:54.619197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:54.620812 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:54.622262 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.622269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.622271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.622496 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.622672 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.622673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.622673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.622786 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.623016 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:54.623022 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:54.623149 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.623235 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.623414 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260809 16:34:54.624255 6404 heartbeater.cc:347] Connected to a master server at 127.31.15.254:42721
I20260809 16:34:54.624341 6404 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:54.624483 6404 heartbeater.cc:511] Master 127.31.15.254:42721 requested a full tablet report, sending...
I20260809 16:34:54.624962 6102 ts_manager.cc:194] Registered new tserver with Master: bc59b533ccf0430cb10db3c0a941ef1a (127.31.15.193:46253)
I20260809 16:34:54.625545 6102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:38821
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:54.643262 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:54.722065 6412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:54.722261 6412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:54.722293 6412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:54.723439 6412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:54.723479 6412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:54.723515 6412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:54.725015 6412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:54.725252 6412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:54.725447 6412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:54.730011 6426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.730011 6423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:54.730091 6412 server_base.cc:1034] running on GCE node
W20260809 16:34:54.730012 6421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:54.730373 6412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:54.730587 6412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:54.731719 6412 hybrid_clock.cc:648] HybridClock initialized: now 1786293294731703 us; error 32 us; skew 500 ppm
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:54.734095 6412 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:54.734370 6412 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:54.734413 6412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:54.734504 6412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:54.735514 6412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "36e49c72be2547c7a2bcafd0984c2c19"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "354403056d9c8d81416e5ea490d49bac"
server_key_iv: "0d286fc9b7ee22a94336dac0412ca8d4"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.735806 6412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "36e49c72be2547c7a2bcafd0984c2c19"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "354403056d9c8d81416e5ea490d49bac"
server_key_iv: "0d286fc9b7ee22a94336dac0412ca8d4"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.737092 6412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:54.737701 6439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.737846 6412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:54.737924 6412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "36e49c72be2547c7a2bcafd0984c2c19"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "354403056d9c8d81416e5ea490d49bac"
server_key_iv: "0d286fc9b7ee22a94336dac0412ca8d4"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.737984 6412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:54.748162 6412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:54.750078 6412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:54.750180 6412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:54.750344 6412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:54.750582 6412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:54.750612 6412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.750630 6412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:54.750645 6412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.755841 6412 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:41471
I20260809 16:34:54.755898 6605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:41471 every 8 connection(s)
I20260809 16:34:54.756115 6412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:54.757316 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 6412
I20260809 16:34:54.757407 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:54.757608 31807 external_mini_cluster.cc:1428] Setting key 1f6e292f47b6a7ab6b44748ebafeb186
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:54.765199 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.756753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:51871 (local address 127.31.15.254:42721)
0809 16:34:54.756830 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.756832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.756972 (+ 140us) server_negotiation.cc:407] Connection header received
0809 16:34:54.757446 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.757447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.757460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.757522 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:54.758557 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.759203 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.760380 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.760573 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.760620 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.762562 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:54.762563 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:54.762564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:54.762571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:54.763993 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.764334 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.764335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.764336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.764395 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.764605 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.764606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.764606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.764733 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:54.764809 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.765025 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.765106 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:34:54.765250 6609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.756723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:42721 (local address 127.31.15.194:51871)
0809 16:34:54.756861 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.756895 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.757349 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.757620 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.757624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.757953 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:54.758475 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.758481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.759309 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.759310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.760292 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.760295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.760639 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.760640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.760771 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.761175 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:54.761184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:54.762476 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:54.764067 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.764073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.764074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.764270 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.764461 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.764462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.764462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.764547 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.764796 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:54.764800 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:54.764904 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.764985 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.765098 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260809 16:34:54.765712 6606 heartbeater.cc:347] Connected to a master server at 127.31.15.254:42721
I20260809 16:34:54.765799 6606 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:54.765933 6606 heartbeater.cc:511] Master 127.31.15.254:42721 requested a full tablet report, sending...
I20260809 16:34:54.766299 6102 ts_manager.cc:194] Registered new tserver with Master: 36e49c72be2547c7a2bcafd0984c2c19 (127.31.15.194:41471)
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
I20260809 16:34:54.766695 6102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:51871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:54.781571 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:54.860181 6615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:54.860365 6615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:54.860397 6615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:54.861521 6615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:54.861562 6615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:54.861595 6615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:54.863032 6615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:42721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:54.863251 6615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:54.863477 6615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:54.867966 6622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.868012 6623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:54.867990 6628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:54.868252 6615 server_base.cc:1034] running on GCE node
I20260809 16:34:54.868476 6615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:54.868685 6615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:54.869820 6615 hybrid_clock.cc:648] HybridClock initialized: now 1786293294869809 us; error 25 us; skew 500 ppm
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:54.872154 6615 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:54.872469 6615 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:54.872522 6615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:54.872643 6615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:54.873611 6615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "b77751a0ff544087a43128ebc2a642fb"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "126082471101f74bbe685310a3db1be8"
server_key_iv: "d8a42e97a29eb85e99480b3ef9f7c599"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.873924 6615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "b77751a0ff544087a43128ebc2a642fb"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "126082471101f74bbe685310a3db1be8"
server_key_iv: "d8a42e97a29eb85e99480b3ef9f7c599"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.875082 6615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:54.875801 6635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.875947 6615 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.876034 6615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "b77751a0ff544087a43128ebc2a642fb"
format_stamp: "Formatted at 2026-08-09 16:34:54 on dist-test-slave-21b9"
server_key: "126082471101f74bbe685310a3db1be8"
server_key_iv: "d8a42e97a29eb85e99480b3ef9f7c599"
server_key_version: "encryptionkey@0"
I20260809 16:34:54.876097 6615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:54.895229 6615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:54.896947 6615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:54.897081 6615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:54.897281 6615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:54.897583 6615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:54.897616 6615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.897636 6615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:54.897660 6615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:54.903067 6615 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:39097
I20260809 16:34:54.903118 6805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:39097 every 8 connection(s)
I20260809 16:34:54.903391 6615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:54.905781 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 6615
I20260809 16:34:54.905889 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:54.906073 31807 external_mini_cluster.cc:1428] Setting key 384aa86d3b2bdd619442793a89f131c2
Aug 09 16:34:54 dist-test-slave-21b9 krb5kdc[6034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293294, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:54.911689 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.904058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:59097 (local address 127.31.15.254:42721)
0809 16:34:54.904115 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.904117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.904277 (+ 160us) server_negotiation.cc:407] Connection header received
0809 16:34:54.904721 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.904722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.904732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.904782 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:54.905588 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.906120 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.907149 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.907360 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.907413 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.909414 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:54.909416 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:54.909416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:54.909422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:54.910663 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.910967 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.910968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.910968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.911013 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.911186 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.911187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.911187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.911304 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:54.911356 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.911569 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.911627 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:54.911859 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.904018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:42721 (local address 127.31.15.195:59097)
0809 16:34:54.904163 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.904202 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.904649 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.904832 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.904836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.905081 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:54.905520 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.905525 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.906200 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.906201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.907073 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.907076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.907425 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.907426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.907530 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.907975 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:54.907984 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:54.909326 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:54.910723 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.910726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.910728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.910914 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.911061 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:54.911061 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:54.911061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:54.911146 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:54.911356 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:54.911360 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:54.911468 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.911548 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.911663 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260809 16:34:54.912258 6808 heartbeater.cc:347] Connected to a master server at 127.31.15.254:42721
I20260809 16:34:54.912346 6808 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:54.912487 6808 heartbeater.cc:511] Master 127.31.15.254:42721 requested a full tablet report, sending...
I20260809 16:34:54.912818 6102 ts_manager.cc:194] Registered new tserver with Master: b77751a0ff544087a43128ebc2a642fb (127.31.15.195:39097)
I20260809 16:34:54.913168 6102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:59097
I20260809 16:34:54.917539 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:34:54.923905 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.918739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.31.15.254:42721)
0809 16:34:54.918783 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.918784 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.918896 (+ 112us) server_negotiation.cc:407] Connection header received
0809 16:34:54.918936 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.918937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.918944 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.918987 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:54.919553 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.920011 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.920688 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.920877 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.920933 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.921929 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:54.921930 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:54.921930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:54.921936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:54.922996 (+ 1060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.923302 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.923303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.923303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.923355 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:54.923494 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:54.923495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:54.923495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:54.923623 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:54.923692 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.923787 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.923847 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:54.925326 6102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:54.926858 6102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:54.932804 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.929107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39097 (local address 127.0.0.1:54746)
0809 16:34:54.929387 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.929396 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.929449 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.929686 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.929687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.929688 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:54.929737 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.929739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.930599 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.930600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.931787 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.931790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932532 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.932608 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.932622 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.932684 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.932733 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":87,"threads_started":1}
I20260809 16:34:54.932814 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.929224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.31.15.195:39097)
0809 16:34:54.929375 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.929381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.929439 (+ 58us) server_negotiation.cc:407] Connection header received
0809 16:34:54.929518 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.929520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.929535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.929603 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:54.929810 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.930510 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.931874 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932460 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932521 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.932592 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.932697 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.932736 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260809 16:34:54.933135 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.928792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:41471 (local address 127.0.0.1:45260)
0809 16:34:54.928901 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.928940 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.929001 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.929836 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.929837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.929839 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:54.929871 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.929872 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.931019 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.931021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.932036 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.932039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932912 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.932987 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.932990 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.933020 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.933058 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
I20260809 16:34:54.933148 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.928988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46253 (local address 127.0.0.1:60390)
0809 16:34:54.929095 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:54.929106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:54.929170 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:54.929453 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:54.929456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:54.929459 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:54.929596 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.929599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.930944 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.930945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.932006 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:54.932008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932961 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932961 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:54.933011 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.933014 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:54.933054 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 16:34:54.933083 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":11,"thread_start_us":43,"threads_started":1}
I20260809 16:34:54.933312 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.929118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60390 (local address 127.31.15.193:46253)
0809 16:34:54.929310 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.929314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.929321 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:34:54.929352 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.929354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.929366 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.929435 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:54.929682 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.930847 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932077 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932811 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932879 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.932933 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.933187 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.933232 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260809 16:34:54.933782 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:54.928908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.31.15.194:41471)
0809 16:34:54.929589 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:54.929593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:54.929603 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:34:54.929638 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:54.929640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:54.929733 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:54.929834 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:54.929920 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.930939 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.932118 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:54.932811 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:54.933602 (+ 791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:54.933658 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:54.933679 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 16:34:54.933711 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":93,"threads_started":1}
I20260809 16:34:54.933882 6713 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 (DEFAULT_TABLE table=test-table [id=6f3a49503a9e4008b2ee38ed2ceabec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:54.933888 6299 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 (DEFAULT_TABLE table=test-table [id=6f3a49503a9e4008b2ee38ed2ceabec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:54.934249 6299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9be2218fa40bab5e2e41a4e4d5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:54.934247 6713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9be2218fa40bab5e2e41a4e4d5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:54.937522 6832 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: Bootstrap starting.
I20260809 16:34:54.938004 6515 tablet_service.cc:1467] Processing CreateTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 (DEFAULT_TABLE table=test-table [id=6f3a49503a9e4008b2ee38ed2ceabec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:54.938143 6832 tablet_bootstrap.cc:654] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:54.938308 6515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d8b9be2218fa40bab5e2e41a4e4d5f84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:54.938647 6832 log.cc:824] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:54.938941 6831 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Bootstrap starting.
I20260809 16:34:54.939390 6832 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: No bootstrap required, opened a new log
I20260809 16:34:54.939463 6832 ts_tablet_manager.cc:1397] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:54.939483 6831 tablet_bootstrap.cc:654] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:54.939931 6831 log.cc:824] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:54.940798 6831 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: No bootstrap required, opened a new log
I20260809 16:34:54.940857 6831 ts_tablet_manager.cc:1397] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:34:54.941540 6840 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Bootstrap starting.
I20260809 16:34:54.941444 6832 raft_consensus.cc:348] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.941619 6832 raft_consensus.cc:374] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:54.941670 6832 raft_consensus.cc:729] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc59b533ccf0430cb10db3c0a941ef1a, State: Initialized, Role: FOLLOWER
I20260809 16:34:54.941807 6832 consensus_queue.cc:260] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.942070 6832 ts_tablet_manager.cc:1428] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 16:34:54.942090 6404 heartbeater.cc:503] Master 127.31.15.254:42721 was elected leader, sending a full tablet report...
I20260809 16:34:54.942237 6840 tablet_bootstrap.cc:654] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:54.942698 6831 raft_consensus.cc:348] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.942801 6840 log.cc:824] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:54.942857 6831 raft_consensus.cc:374] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:54.942924 6831 raft_consensus.cc:729] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b77751a0ff544087a43128ebc2a642fb, State: Initialized, Role: FOLLOWER
I20260809 16:34:54.943097 6831 consensus_queue.cc:260] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.943385 6831 ts_tablet_manager.cc:1428] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:34:54.943408 6808 heartbeater.cc:503] Master 127.31.15.254:42721 was elected leader, sending a full tablet report...
I20260809 16:34:54.943572 6840 tablet_bootstrap.cc:492] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: No bootstrap required, opened a new log
I20260809 16:34:54.943630 6840 ts_tablet_manager.cc:1397] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:54.945223 6840 raft_consensus.cc:348] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.945335 6840 raft_consensus.cc:374] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:54.945355 6840 raft_consensus.cc:729] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e49c72be2547c7a2bcafd0984c2c19, State: Initialized, Role: FOLLOWER
I20260809 16:34:54.945461 6840 consensus_queue.cc:260] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:54.945690 6840 ts_tablet_manager.cc:1428] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:34:54.945694 6606 heartbeater.cc:503] Master 127.31.15.254:42721 was elected leader, sending a full tablet report...
I20260809 16:34:55.002676 6843 raft_consensus.cc:482] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:55.002803 6843 raft_consensus.cc:504] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:55.003145 6843 leader_election.cc:288] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36e49c72be2547c7a2bcafd0984c2c19 (127.31.15.194:41471), bc59b533ccf0430cb10db3c0a941ef1a (127.31.15.193:46253)
W20260809 16:34:55.006697 6607 tablet.cc:2369] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:55.006891 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.003315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:41471 (local address 127.31.15.195:39457)
0809 16:34:55.003425 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.003439 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.003495 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.003648 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.003649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.003651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:55.003725 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.003727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.004617 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.004617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.005906 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.005909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.006625 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.006626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.006714 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.006718 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.006776 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.006819 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":69}
I20260809 16:34:55.006904 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.003367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:39457 (local address 127.31.15.194:41471)
0809 16:34:55.003436 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.003438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.003539 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:34:55.003565 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.003565 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.003574 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.003627 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:55.003791 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.004546 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.005986 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.006544 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.006619 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.006627 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.006798 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.006843 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:34:55.006937 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.003534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:45113 (local address 127.31.15.193:46253)
0809 16:34:55.003606 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.003608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.003673 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:34:55.003733 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.003734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.003741 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.003782 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:34:55.003902 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.004580 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.005986 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.006544 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.006618 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.006627 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.006837 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.006881 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 16:34:55.007169 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.003417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46253 (local address 127.31.15.195:45113)
0809 16:34:55.003607 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.003617 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.003679 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.003796 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.003796 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.003798 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:34:55.003848 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.003849 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.004640 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.004641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.005906 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.005909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.006677 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.006678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.006750 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.006754 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.007030 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.007084 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2,"thread_start_us":121,"threads_started":1}
I20260809 16:34:55.007232 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9be2218fa40bab5e2e41a4e4d5f84" candidate_uuid: "b77751a0ff544087a43128ebc2a642fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36e49c72be2547c7a2bcafd0984c2c19" is_pre_election: true
I20260809 16:34:55.007411 6542 raft_consensus.cc:2454] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b77751a0ff544087a43128ebc2a642fb in term 0.
I20260809 16:34:55.007583 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9be2218fa40bab5e2e41a4e4d5f84" candidate_uuid: "b77751a0ff544087a43128ebc2a642fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" is_pre_election: true
I20260809 16:34:55.007655 6643 leader_election.cc:302] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e49c72be2547c7a2bcafd0984c2c19, b77751a0ff544087a43128ebc2a642fb; no voters:
I20260809 16:34:55.007705 6331 raft_consensus.cc:2454] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b77751a0ff544087a43128ebc2a642fb in term 0.
I20260809 16:34:55.007793 6843 raft_consensus.cc:2781] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:55.007845 6843 raft_consensus.cc:482] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:55.007869 6843 raft_consensus.cc:3037] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.008611 6843 raft_consensus.cc:504] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:55.008715 6843 leader_election.cc:288] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [CANDIDATE]: Term 1 election: Requested vote from peers 36e49c72be2547c7a2bcafd0984c2c19 (127.31.15.194:41471), bc59b533ccf0430cb10db3c0a941ef1a (127.31.15.193:46253)
I20260809 16:34:55.008893 6542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9be2218fa40bab5e2e41a4e4d5f84" candidate_uuid: "b77751a0ff544087a43128ebc2a642fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36e49c72be2547c7a2bcafd0984c2c19"
I20260809 16:34:55.008927 6331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8b9be2218fa40bab5e2e41a4e4d5f84" candidate_uuid: "b77751a0ff544087a43128ebc2a642fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc59b533ccf0430cb10db3c0a941ef1a"
I20260809 16:34:55.008992 6542 raft_consensus.cc:3037] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.008998 6331 raft_consensus.cc:3037] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.009886 6331 raft_consensus.cc:2454] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b77751a0ff544087a43128ebc2a642fb in term 1.
I20260809 16:34:55.009886 6542 raft_consensus.cc:2454] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b77751a0ff544087a43128ebc2a642fb in term 1.
I20260809 16:34:55.010051 6643 leader_election.cc:302] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e49c72be2547c7a2bcafd0984c2c19, b77751a0ff544087a43128ebc2a642fb; no voters:
I20260809 16:34:55.010136 6843 raft_consensus.cc:2781] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:55.010285 6843 raft_consensus.cc:686] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 1 LEADER]: Becoming Leader. State: Replica: b77751a0ff544087a43128ebc2a642fb, State: Running, Role: LEADER
I20260809 16:34:55.010371 6843 consensus_queue.cc:237] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } }
I20260809 16:34:55.011181 6102 catalog_manager.cc:5495] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb reported cstate change: term changed from 0 to 1, leader changed from <none> to b77751a0ff544087a43128ebc2a642fb (127.31.15.195). New cstate: current_term: 1 leader_uuid: "b77751a0ff544087a43128ebc2a642fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b77751a0ff544087a43128ebc2a642fb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39097 } health_report { overall_health: HEALTHY } } }
I20260809 16:34:55.022578 6830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.020654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.31.15.195:39097)
0809 16:34:55.020696 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.020697 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.020705 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:55.020765 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.020765 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.020780 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.020840 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:34:55.020925 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.021374 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.022043 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.022209 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.022249 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.022340 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:34:55.022432 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:34:55.022484 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.022500 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.022514 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:34:55.024209 6331 raft_consensus.cc:1261] T d8b9be2218fa40bab5e2e41a4e4d5f84 P bc59b533ccf0430cb10db3c0a941ef1a [term 1 FOLLOWER]: Refusing update from remote peer b77751a0ff544087a43128ebc2a642fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:55.024209 6542 raft_consensus.cc:1261] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 1 FOLLOWER]: Refusing update from remote peer b77751a0ff544087a43128ebc2a642fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:55.024539 6843 consensus_queue.cc:1035] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc59b533ccf0430cb10db3c0a941ef1a" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:55.024667 6847 consensus_queue.cc:1035] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "36e49c72be2547c7a2bcafd0984c2c19" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:55.026911 6853 mvcc.cc:205] Tried to move back new op lower bound from 7316657336416903168 to 7316657336363134976. Current Snapshot: MvccSnapshot[applied={T|T < 7316657336416903168}]
I20260809 16:34:55.027024 6857 mvcc.cc:205] Tried to move back new op lower bound from 7316657336416903168 to 7316657336363134976. Current Snapshot: MvccSnapshot[applied={T|T < 7316657336416903168}]
I20260809 16:34:55.027499 6856 mvcc.cc:205] Tried to move back new op lower bound from 7316657336416903168 to 7316657336363134976. Current Snapshot: MvccSnapshot[applied={T|T < 7316657336416903168}]
I20260809 16:34:55.028971 6097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:55.029050 6097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:55.029780 6097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27cd07e2064f46578280d0c1f158f97b: Sending DeleteTablet for 3 replicas of tablet d8b9be2218fa40bab5e2e41a4e4d5f84
I20260809 16:34:55.030123 6713 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:54746
I20260809 16:34:55.030191 6515 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45260
I20260809 16:34:55.030206 6299 tablet_service.cc:1514] Processing DeleteTablet for tablet d8b9be2218fa40bab5e2e41a4e4d5f84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60390
I20260809 16:34:55.030434 6867 tablet_replica.cc:330] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: stopping tablet replica
I20260809 16:34:55.030514 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 6193
I20260809 16:34:55.030579 6867 raft_consensus.cc:2229] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:55.030678 6867 raft_consensus.cc:2258] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:55.031068 6867 ts_tablet_manager.cc:1905] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:55.033107 6867 ts_tablet_manager.cc:1918] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:55.033216 6867 log.cc:1196] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/d8b9be2218fa40bab5e2e41a4e4d5f84
I20260809 16:34:55.033574 6867 ts_tablet_manager.cc:1939] T d8b9be2218fa40bab5e2e41a4e4d5f84 P 36e49c72be2547c7a2bcafd0984c2c19: Deleting consensus metadata
I20260809 16:34:55.034065 6082 catalog_manager.cc:4841] TS 36e49c72be2547c7a2bcafd0984c2c19 (127.31.15.194:41471): tablet d8b9be2218fa40bab5e2e41a4e4d5f84 (table test-table [id=6f3a49503a9e4008b2ee38ed2ceabec8]) successfully deleted
I20260809 16:34:55.038662 6866 tablet_replica.cc:330] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: stopping tablet replica
I20260809 16:34:55.038815 6866 raft_consensus.cc:2229] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:34:55.038975 6866 raft_consensus.cc:2258] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:55.039448 6866 ts_tablet_manager.cc:1905] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:55.041420 6866 ts_tablet_manager.cc:1918] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:55.041559 6866 log.cc:1196] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/d8b9be2218fa40bab5e2e41a4e4d5f84
I20260809 16:34:55.041918 6866 ts_tablet_manager.cc:1939] T d8b9be2218fa40bab5e2e41a4e4d5f84 P b77751a0ff544087a43128ebc2a642fb: Deleting consensus metadata
I20260809 16:34:55.042361 6082 catalog_manager.cc:4841] TS b77751a0ff544087a43128ebc2a642fb (127.31.15.195:39097): tablet d8b9be2218fa40bab5e2e41a4e4d5f84 (table test-table [id=6f3a49503a9e4008b2ee38ed2ceabec8]) successfully deleted
W20260809 16:34:55.045677 6082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:46253 (error 108)
I20260809 16:34:55.045897 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 6412
I20260809 16:34:55.047840 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.046091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46253 (local address 127.0.0.1:60400)
0809 16:34:55.047746 (+ 1655us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.047803 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1626}
W20260809 16:34:55.050467 6082 catalog_manager.cc:4584] TS bc59b533ccf0430cb10db3c0a941ef1a (127.31.15.193:46253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d8b9be2218fa40bab5e2e41a4e4d5f84: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46253: connect: Connection refused (error 111)
I20260809 16:34:55.056282 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 6615
I20260809 16:34:55.068308 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 6053
2026-08-09T16:34:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2047 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T16:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:55Z Disabled control of system clock
I20260809 16:34:55.101660 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45279
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:45279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:34:55.180570 6876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:55.180758 6876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:55.180778 6876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:55.181872 6876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:55.181913 6876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:55.181924 6876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:55.181936 6876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:55.181947 6876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:55.183262 6876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:45279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:55.183496 6876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:55.183764 6876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:55.188166 6884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.188171 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.188287 6876 server_base.cc:1034] running on GCE node
W20260809 16:34:55.188203 6888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.188607 6876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:55.188843 6876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:55.190001 6876 hybrid_clock.cc:648] HybridClock initialized: now 1786293295189977 us; error 37 us; skew 500 ppm
I20260809 16:34:55.191145 6876 webserver.cc:460] Webserver started at http://127.31.15.254:33305/ using document root <none> and password file <none>
I20260809 16:34:55.191377 6876 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:55.191426 6876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:55.191568 6876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:55.192576 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "dee284322de04423b00c29a773b47128"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "7df9680408059d202e36a7106ff3ba9f"
server_key_iv: "83a415fee5c40f2a41aa24fcabb12e41"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.192893 6876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "dee284322de04423b00c29a773b47128"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "7df9680408059d202e36a7106ff3ba9f"
server_key_iv: "83a415fee5c40f2a41aa24fcabb12e41"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.194123 6876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:55.194738 6895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.194885 6876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:55.194968 6876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "dee284322de04423b00c29a773b47128"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "7df9680408059d202e36a7106ff3ba9f"
server_key_iv: "83a415fee5c40f2a41aa24fcabb12e41"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.195029 6876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:55.198195 6876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:55.198446 6876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:55.198572 6876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:55.202322 6876 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:45279
I20260809 16:34:55.202354 6979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:45279 every 8 connection(s)
I20260809 16:34:55.202639 6876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:55.203233 6980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:55.205612 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 6876
I20260809 16:34:55.205724 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:55.205902 31807 external_mini_cluster.cc:1428] Setting key 57d3422e222fb70a041c8d3a45d990b5
I20260809 16:34:55.206043 6980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: Bootstrap starting.
I20260809 16:34:55.206689 6980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:55.206974 6980 log.cc:824] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:55.207984 6980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: No bootstrap required, opened a new log
I20260809 16:34:55.208225 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.206373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45498 (local address 127.31.15.254:45279)
0809 16:34:55.206556 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.206563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.206587 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:34:55.207016 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.207023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.207076 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.207210 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.207827 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.207830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.207830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.207833 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.207867 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.207876 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.207934 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.208005 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.208052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260809 16:34:55.209530 6980 raft_consensus.cc:348] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } }
I20260809 16:34:55.209656 6980 raft_consensus.cc:374] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:55.209676 6980 raft_consensus.cc:729] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dee284322de04423b00c29a773b47128, State: Initialized, Role: FOLLOWER
I20260809 16:34:55.209769 6980 consensus_queue.cc:260] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } }
I20260809 16:34:55.210103 6980 raft_consensus.cc:388] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:55.210134 6980 raft_consensus.cc:482] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:55.210165 6980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.210600 6980 raft_consensus.cc:504] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } }
I20260809 16:34:55.210683 6980 leader_election.cc:302] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dee284322de04423b00c29a773b47128; no voters:
I20260809 16:34:55.210793 6980 leader_election.cc:288] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:55.210852 6985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:55.210963 6985 raft_consensus.cc:686] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [term 1 LEADER]: Becoming Leader. State: Replica: dee284322de04423b00c29a773b47128, State: Running, Role: LEADER
I20260809 16:34:55.211061 6980 sys_catalog.cc:564] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:55.211122 6985 consensus_queue.cc:237] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } }
I20260809 16:34:55.211539 6986 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dee284322de04423b00c29a773b47128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } } }
I20260809 16:34:55.211570 6987 sys_catalog.cc:455] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dee284322de04423b00c29a773b47128. Latest consensus state: current_term: 1 leader_uuid: "dee284322de04423b00c29a773b47128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dee284322de04423b00c29a773b47128" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45279 } } }
I20260809 16:34:55.211685 6987 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:55.212071 6986 sys_catalog.cc:458] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:55.212148 6995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:55.212591 6995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:55.213955 6995 catalog_manager.cc:1347] Generated new cluster ID: c35adfd5465d49479bc8df54fdf3b09c
I20260809 16:34:55.213997 6995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:55.234992 6995 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:55.235149 6995 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:55.240973 6995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: Generated new TSK 0
I20260809 16:34:55.241128 6995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:55.251935 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--builtin_ntp_servers=127.31.15.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:34:55.355841 7014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:55.356030 7014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:55.356050 7014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:55.357187 7014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:55.357227 7014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:55.357252 7014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:55.358554 7014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:55.358790 7014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:55.359050 7014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:55.363591 7021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.363581 7025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.363579 7022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.363870 7014 server_base.cc:1034] running on GCE node
I20260809 16:34:55.364058 7014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:55.364275 7014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:55.365411 7014 hybrid_clock.cc:648] HybridClock initialized: now 1786293295365393 us; error 33 us; skew 500 ppm
I20260809 16:34:55.366539 7014 webserver.cc:460] Webserver started at http://127.31.15.193:32801/ using document root <none> and password file <none>
I20260809 16:34:55.366745 7014 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:55.366787 7014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:55.366909 7014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:55.367930 7014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "e4910fba32354262bcb64ac66d324215"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "79f53f0111c697699ab3497109a6cc92"
server_key_iv: "e8ec59f473b6c3a1d2be0221f37d9968"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.368242 7014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "e4910fba32354262bcb64ac66d324215"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "79f53f0111c697699ab3497109a6cc92"
server_key_iv: "e8ec59f473b6c3a1d2be0221f37d9968"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.369478 7014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:55.370095 7038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.370229 7014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:55.370309 7014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "e4910fba32354262bcb64ac66d324215"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "79f53f0111c697699ab3497109a6cc92"
server_key_iv: "e8ec59f473b6c3a1d2be0221f37d9968"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.370368 7014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:55.373541 7014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:55.373757 7014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:55.373867 7014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:55.374053 7014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:55.374333 7014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:55.374365 7014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.374389 7014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:55.374416 7014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.379721 7014 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:38313
I20260809 16:34:55.379765 7220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:38313 every 8 connection(s)
I20260809 16:34:55.380043 7014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:34:55.382570 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.380677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:59225 (local address 127.31.15.254:45279)
0809 16:34:55.380738 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.380740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.380964 (+ 224us) server_negotiation.cc:407] Connection header received
0809 16:34:55.381362 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.381363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.381368 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.381421 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.382393 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.382394 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.382394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.382398 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.382413 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.382417 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.382474 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.382504 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.382516 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:55.382750 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.380643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45279 (local address 127.31.15.193:59225)
0809 16:34:55.380829 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.380867 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.381273 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.381467 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.381470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.381477 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.382268 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.382268 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.382293 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.382319 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.382457 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.382459 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.382461 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.382512 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.382557 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260809 16:34:55.383179 7221 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45279
I20260809 16:34:55.383262 7221 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:55.383428 7221 heartbeater.cc:511] Master 127.31.15.254:45279 requested a full tablet report, sending...
I20260809 16:34:55.383909 6925 ts_manager.cc:194] Registered new tserver with Master: e4910fba32354262bcb64ac66d324215 (127.31.15.193:38313)
I20260809 16:34:55.387797 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 7014
I20260809 16:34:55.387882 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:55.388067 31807 external_mini_cluster.cc:1428] Setting key 53df152b3becbd43b099635b238ce6b8
I20260809 16:34:55.388902 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--builtin_ntp_servers=127.31.15.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 16:34:55.468166 7226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:55.468350 7226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:55.468370 7226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:55.469465 7226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:55.469498 7226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:55.469521 7226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:55.470783 7226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:55.470971 7226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:55.471220 7226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:55.475847 7231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.475862 7232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.475862 7234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.475996 7226 server_base.cc:1034] running on GCE node
I20260809 16:34:55.476276 7226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:55.476506 7226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:55.477641 7226 hybrid_clock.cc:648] HybridClock initialized: now 1786293295477629 us; error 28 us; skew 500 ppm
I20260809 16:34:55.478811 7226 webserver.cc:460] Webserver started at http://127.31.15.194:33023/ using document root <none> and password file <none>
I20260809 16:34:55.479034 7226 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:55.479079 7226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:55.479210 7226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:55.480279 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "ca319b4bcc3a460c89819c20fdfb1543"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "8ca978df566af793bf528ec468f1b886"
server_key_iv: "294e2d2d929135fbad6e9ed23203cd01"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.480610 7226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "ca319b4bcc3a460c89819c20fdfb1543"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "8ca978df566af793bf528ec468f1b886"
server_key_iv: "294e2d2d929135fbad6e9ed23203cd01"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.481771 7226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:55.482394 7244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.482527 7226 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:55.482605 7226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "ca319b4bcc3a460c89819c20fdfb1543"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "8ca978df566af793bf528ec468f1b886"
server_key_iv: "294e2d2d929135fbad6e9ed23203cd01"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.482669 7226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:55.485945 7226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:55.486177 7226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:55.486291 7226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:55.486475 7226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:55.486763 7226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:55.486797 7226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.486816 7226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:55.486840 7226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.492231 7226 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:38191
I20260809 16:34:55.492267 7438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:38191 every 8 connection(s)
I20260809 16:34:55.492554 7226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:55.493594 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 7226
I20260809 16:34:55.493686 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:55.493870 31807 external_mini_cluster.cc:1428] Setting key a68352f57c40ddb99578a4ee42db92ac
I20260809 16:34:55.494705 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--builtin_ntp_servers=127.31.15.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 16:34:55.495430 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.493211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:58179 (local address 127.31.15.254:45279)
0809 16:34:55.493275 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.493277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.493487 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:34:55.493979 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.493980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.493987 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.494045 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.495183 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.495184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.495185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.495190 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.495205 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.495209 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.495257 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.495348 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.495361 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:34:55.495659 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.493182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45279 (local address 127.31.15.194:58179)
0809 16:34:55.493367 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.493407 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.493884 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.494112 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.494117 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.494126 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.495047 (+ 921us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.495048 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.495076 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.495103 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.495263 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.495286 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.495289 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.495359 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.495420 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260809 16:34:55.496078 7440 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45279
I20260809 16:34:55.496165 7440 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:55.496306 7440 heartbeater.cc:511] Master 127.31.15.254:45279 requested a full tablet report, sending...
I20260809 16:34:55.496667 6925 ts_manager.cc:194] Registered new tserver with Master: ca319b4bcc3a460c89819c20fdfb1543 (127.31.15.194:38191)
W20260809 16:34:55.576519 7446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:55.576723 7446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:55.576753 7446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:55.577867 7446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:55.577904 7446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:55.577936 7446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:55.579245 7446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:55.579517 7446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:55.579828 7446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:55.584479 7453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.584501 7452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.584620 7446 server_base.cc:1034] running on GCE node
W20260809 16:34:55.584508 7456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.584867 7446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:55.585144 7446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:55.586298 7446 hybrid_clock.cc:648] HybridClock initialized: now 1786293295586286 us; error 27 us; skew 500 ppm
I20260809 16:34:55.587536 7446 webserver.cc:460] Webserver started at http://127.31.15.195:33179/ using document root <none> and password file <none>
I20260809 16:34:55.587733 7446 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:55.587772 7446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:55.587865 7446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:55.588850 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "c4622e86e5d046a0a1add23eeaf8b793"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "3e4a216b0fdb7e8bf50e3a1accbe16b5"
server_key_iv: "451ccd9c30fb91dcf4ba5bed1ac6de0a"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.589154 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "c4622e86e5d046a0a1add23eeaf8b793"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "3e4a216b0fdb7e8bf50e3a1accbe16b5"
server_key_iv: "451ccd9c30fb91dcf4ba5bed1ac6de0a"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.590361 7446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:55.590970 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.591115 7446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:55.591198 7446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "c4622e86e5d046a0a1add23eeaf8b793"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "3e4a216b0fdb7e8bf50e3a1accbe16b5"
server_key_iv: "451ccd9c30fb91dcf4ba5bed1ac6de0a"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.591256 7446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:55.594422 7446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:55.594671 7446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:55.594784 7446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:55.594967 7446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:55.595247 7446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:55.595304 7446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.595331 7446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:55.595355 7446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.600955 7446 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:34769
I20260809 16:34:55.601013 7642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:34769 every 8 connection(s)
I20260809 16:34:55.601289 7446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:55.603864 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.601893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:58543 (local address 127.31.15.254:45279)
0809 16:34:55.601955 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.601958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.602178 (+ 220us) server_negotiation.cc:407] Connection header received
0809 16:34:55.602613 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.602615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.602620 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.602656 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.603696 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.603697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.603698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.603701 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.603715 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.603719 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.603756 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.603800 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.603811 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:34:55.604027 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.601886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45279 (local address 127.31.15.195:58543)
0809 16:34:55.602069 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.602106 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.602519 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.602716 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.602719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.602727 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.603563 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.603564 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.603590 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.603617 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.603756 (+ 139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.603758 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.603760 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.603806 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.603850 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260809 16:34:55.604494 7643 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45279
I20260809 16:34:55.604578 7643 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:55.604715 7643 heartbeater.cc:511] Master 127.31.15.254:45279 requested a full tablet report, sending...
I20260809 16:34:55.605046 6925 ts_manager.cc:194] Registered new tserver with Master: c4622e86e5d046a0a1add23eeaf8b793 (127.31.15.195:34769)
I20260809 16:34:55.610248 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 7446
I20260809 16:34:55.610349 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:55.610518 31807 external_mini_cluster.cc:1428] Setting key 14600b4125f154a1df241030e6943c9f
I20260809 16:34:55.611348 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:34:55.613582 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.612733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.31.15.254:45279)
0809 16:34:55.612808 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.612810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.612873 (+ 63us) server_negotiation.cc:407] Connection header received
0809 16:34:55.612920 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.612921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.612925 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.612968 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.613400 (+ 432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.613400 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.613401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.613404 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.613414 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.613417 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.613473 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.613506 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.613516 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:34:55.614853 6925 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:34:55.616468 6925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:34:55.620087 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:38313 (local address 127.0.0.1:33376)
0809 16:34:55.618590 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.618619 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.618672 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.619083 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.619084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.619087 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.619544 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.619545 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.619559 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.619580 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.619937 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.619945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.619948 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.620004 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.620017 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":75,"threads_started":1}
I20260809 16:34:55.620407 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33376 (local address 127.31.15.193:38313)
0809 16:34:55.618950 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.618954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.618963 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:55.618989 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.618991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.619000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.619068 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.619780 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.619790 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.619791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.619795 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.619829 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.619859 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.620278 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.620304 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.620320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":95,"threads_started":1}
I20260809 16:34:55.620472 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:34769 (local address 127.0.0.1:35444)
0809 16:34:55.618871 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.618881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.618933 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.619067 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.619074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.619079 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.619888 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.620082 (+ 194us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.620095 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.620108 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.620337 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.620338 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.620339 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.620389 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.620413 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":11,"thread_start_us":79,"threads_started":1}
I20260809 16:34:55.620587 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.31.15.194:38191)
0809 16:34:55.618912 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.618916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.619407 (+ 491us) server_negotiation.cc:407] Connection header received
0809 16:34:55.619663 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.619671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.619680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.619789 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.620200 (+ 411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.620207 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.620208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.620212 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.620240 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.620263 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.620298 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.620414 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.620431 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260809 16:34:55.620774 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.31.15.195:34769)
0809 16:34:55.618875 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.618880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.618928 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:34:55.618997 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.618999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.619008 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.619066 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.620193 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.620200 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.620201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.620206 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.620240 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.620263 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.620313 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.620688 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.620704 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260809 16:34:55.620474 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.618637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:38191 (local address 127.0.0.1:32966)
0809 16:34:55.619340 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.619349 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.619458 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.619773 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.619774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.619777 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.620097 (+ 320us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.620097 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.620101 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.620108 (+ 7us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.620339 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.620340 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.620340 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.620387 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.620400 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":64,"threads_started":1}
I20260809 16:34:55.621222 7115 tablet_service.cc:1467] Processing CreateTablet for tablet c88046c66e8f491197c26920104f6858 (DEFAULT_TABLE table=test-table [id=f73df1bce00f422ab4e60c9727d62c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:55.621608 7115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88046c66e8f491197c26920104f6858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:55.621626 7545 tablet_service.cc:1467] Processing CreateTablet for tablet c88046c66e8f491197c26920104f6858 (DEFAULT_TABLE table=test-table [id=f73df1bce00f422ab4e60c9727d62c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:55.621659 7329 tablet_service.cc:1467] Processing CreateTablet for tablet c88046c66e8f491197c26920104f6858 (DEFAULT_TABLE table=test-table [id=f73df1bce00f422ab4e60c9727d62c07]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:34:55.621953 7329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88046c66e8f491197c26920104f6858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:55.621953 7545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88046c66e8f491197c26920104f6858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:55.624256 7667 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Bootstrap starting.
I20260809 16:34:55.624851 7667 tablet_bootstrap.cc:654] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:55.625083 7668 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Bootstrap starting.
I20260809 16:34:55.625296 7667 log.cc:824] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:55.625735 7668 tablet_bootstrap.cc:654] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:55.625942 7669 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Bootstrap starting.
I20260809 16:34:55.626174 7667 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: No bootstrap required, opened a new log
I20260809 16:34:55.626250 7667 ts_tablet_manager.cc:1397] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:55.626561 7669 tablet_bootstrap.cc:654] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:55.626873 7669 log.cc:824] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:55.627399 7668 log.cc:824] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:55.627542 7669 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: No bootstrap required, opened a new log
I20260809 16:34:55.627609 7669 ts_tablet_manager.cc:1397] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:34:55.628211 7668 tablet_bootstrap.cc:492] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: No bootstrap required, opened a new log
I20260809 16:34:55.628283 7668 ts_tablet_manager.cc:1397] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:34:55.628729 7667 raft_consensus.cc:348] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.628896 7667 raft_consensus.cc:374] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:55.628980 7667 raft_consensus.cc:729] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4910fba32354262bcb64ac66d324215, State: Initialized, Role: FOLLOWER
I20260809 16:34:55.629112 7667 consensus_queue.cc:260] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.629383 7667 ts_tablet_manager.cc:1428] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:34:55.629541 7221 heartbeater.cc:503] Master 127.31.15.254:45279 was elected leader, sending a full tablet report...
I20260809 16:34:55.629805 7669 raft_consensus.cc:348] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.629956 7669 raft_consensus.cc:374] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:55.630003 7669 raft_consensus.cc:729] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca319b4bcc3a460c89819c20fdfb1543, State: Initialized, Role: FOLLOWER
I20260809 16:34:55.630106 7669 consensus_queue.cc:260] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.630359 7669 ts_tablet_manager.cc:1428] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 16:34:55.630482 7222 tablet.cc:2369] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:34:55.630389 7440 heartbeater.cc:503] Master 127.31.15.254:45279 was elected leader, sending a full tablet report...
I20260809 16:34:55.630640 7668 raft_consensus.cc:348] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.630769 7668 raft_consensus.cc:374] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:55.630801 7668 raft_consensus.cc:729] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4622e86e5d046a0a1add23eeaf8b793, State: Initialized, Role: FOLLOWER
I20260809 16:34:55.630904 7668 consensus_queue.cc:260] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.631140 7668 ts_tablet_manager.cc:1428] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:34:55.631174 7643 heartbeater.cc:503] Master 127.31.15.254:45279 was elected leader, sending a full tablet report...
I20260809 16:34:55.689514 7674 raft_consensus.cc:482] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:34:55.689661 7674 raft_consensus.cc:504] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.690060 7674 leader_election.cc:288] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca319b4bcc3a460c89819c20fdfb1543 (127.31.15.194:38191), c4622e86e5d046a0a1add23eeaf8b793 (127.31.15.195:34769)
I20260809 16:34:55.691241 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.690226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:38191 (local address 127.31.15.193:34303)
0809 16:34:55.690267 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.690280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.690322 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.690579 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.690579 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.690581 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.690950 (+ 369us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.690950 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.690957 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.690967 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.691128 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.691129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.691129 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.691175 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.691189 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:34:55.691288 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.690372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:34303 (local address 127.31.15.194:38191)
0809 16:34:55.690441 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.690442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.690458 (+ 16us) server_negotiation.cc:407] Connection header received
0809 16:34:55.690498 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.690499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.690505 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.690563 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.691046 (+ 483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.691048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.691048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.691052 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.691071 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.691075 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.691120 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.691183 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.691196 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 16:34:55.691398 7662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.690293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:43753 (local address 127.31.15.195:34769)
0809 16:34:55.690348 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.690350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.690481 (+ 131us) server_negotiation.cc:407] Connection header received
0809 16:34:55.690505 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.690506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.690510 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.690566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.691177 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.691178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.691179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.691183 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.691200 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.691204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.691255 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.691325 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.691338 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:34:55.691403 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.690244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:34769 (local address 127.31.15.193:43753)
0809 16:34:55.690410 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.690418 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.690459 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.690715 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.690715 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.690717 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:55.691088 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:34:55.691088 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:55.691093 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:34:55.691101 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:55.691259 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:55.691260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:55.691260 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 16:34:55.691324 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 16:34:55.691335 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":114,"threads_started":1}
I20260809 16:34:55.691598 7362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88046c66e8f491197c26920104f6858" candidate_uuid: "e4910fba32354262bcb64ac66d324215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca319b4bcc3a460c89819c20fdfb1543" is_pre_election: true
I20260809 16:34:55.691725 7362 raft_consensus.cc:2454] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4910fba32354262bcb64ac66d324215 in term 0.
I20260809 16:34:55.691933 7043 leader_election.cc:302] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca319b4bcc3a460c89819c20fdfb1543, e4910fba32354262bcb64ac66d324215; no voters:
I20260809 16:34:55.691932 7575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88046c66e8f491197c26920104f6858" candidate_uuid: "e4910fba32354262bcb64ac66d324215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4622e86e5d046a0a1add23eeaf8b793" is_pre_election: true
I20260809 16:34:55.692063 7575 raft_consensus.cc:2454] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4910fba32354262bcb64ac66d324215 in term 0.
I20260809 16:34:55.692052 7674 raft_consensus.cc:2781] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:34:55.692123 7674 raft_consensus.cc:482] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:34:55.692154 7674 raft_consensus.cc:3037] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.692936 7674 raft_consensus.cc:504] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.693053 7674 leader_election.cc:288] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [CANDIDATE]: Term 1 election: Requested vote from peers ca319b4bcc3a460c89819c20fdfb1543 (127.31.15.194:38191), c4622e86e5d046a0a1add23eeaf8b793 (127.31.15.195:34769)
I20260809 16:34:55.693197 7362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88046c66e8f491197c26920104f6858" candidate_uuid: "e4910fba32354262bcb64ac66d324215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca319b4bcc3a460c89819c20fdfb1543"
I20260809 16:34:55.693233 7575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88046c66e8f491197c26920104f6858" candidate_uuid: "e4910fba32354262bcb64ac66d324215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4622e86e5d046a0a1add23eeaf8b793"
I20260809 16:34:55.693270 7362 raft_consensus.cc:3037] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.693313 7575 raft_consensus.cc:3037] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.693943 7362 raft_consensus.cc:2454] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4910fba32354262bcb64ac66d324215 in term 1.
I20260809 16:34:55.694073 7575 raft_consensus.cc:2454] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4910fba32354262bcb64ac66d324215 in term 1.
I20260809 16:34:55.694097 7043 leader_election.cc:302] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca319b4bcc3a460c89819c20fdfb1543, e4910fba32354262bcb64ac66d324215; no voters:
I20260809 16:34:55.694190 7674 raft_consensus.cc:2781] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:55.694353 7674 raft_consensus.cc:686] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [term 1 LEADER]: Becoming Leader. State: Replica: e4910fba32354262bcb64ac66d324215, State: Running, Role: LEADER
I20260809 16:34:55.694437 7674 consensus_queue.cc:237] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } }
I20260809 16:34:55.695217 6925 catalog_manager.cc:5495] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4910fba32354262bcb64ac66d324215 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "e4910fba32354262bcb64ac66d324215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4910fba32354262bcb64ac66d324215" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 38313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 } health_report { overall_health: UNKNOWN } } }
I20260809 16:34:55.713546 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.712774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.31.15.193:38313)
0809 16:34:55.712813 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.712815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.712844 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:34:55.712921 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.712922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.712933 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.712979 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.713399 (+ 420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.713400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.713400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.713404 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.713416 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.713420 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.713459 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.713486 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.713497 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:34:55.715126 7575 raft_consensus.cc:1261] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 1 FOLLOWER]: Refusing update from remote peer e4910fba32354262bcb64ac66d324215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:55.715126 7362 raft_consensus.cc:1261] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 1 FOLLOWER]: Refusing update from remote peer e4910fba32354262bcb64ac66d324215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:34:55.715497 7679 consensus_queue.cc:1035] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4622e86e5d046a0a1add23eeaf8b793" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 34769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:55.715615 7674 consensus_queue.cc:1035] T c88046c66e8f491197c26920104f6858 P e4910fba32354262bcb64ac66d324215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca319b4bcc3a460c89819c20fdfb1543" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:34:55.717854 7685 mvcc.cc:205] Tried to move back new op lower bound from 7316657339246841856 to 7316657339164987392. Current Snapshot: MvccSnapshot[applied={T|T < 7316657339246841856}]
I20260809 16:34:55.718771 7689 mvcc.cc:205] Tried to move back new op lower bound from 7316657339246841856 to 7316657339164987392. Current Snapshot: MvccSnapshot[applied={T|T < 7316657339246841856 or (T in {7316657339246841856})}]
I20260809 16:34:55.719952 6922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:55.720031 6922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:34:55.720679 6922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dee284322de04423b00c29a773b47128: Sending DeleteTablet for 3 replicas of tablet c88046c66e8f491197c26920104f6858
I20260809 16:34:55.721019 7545 tablet_service.cc:1514] Processing DeleteTablet for tablet c88046c66e8f491197c26920104f6858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='slave'} at 127.0.0.1:35444
I20260809 16:34:55.721017 7329 tablet_service.cc:1514] Processing DeleteTablet for tablet c88046c66e8f491197c26920104f6858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='slave'} at 127.0.0.1:32966
I20260809 16:34:55.721246 7115 tablet_service.cc:1514] Processing DeleteTablet for tablet c88046c66e8f491197c26920104f6858 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:34:55 UTC) from {username='slave'} at 127.0.0.1:33376
I20260809 16:34:55.721374 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 7014
I20260809 16:34:55.721231 7700 tablet_replica.cc:330] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: stopping tablet replica
I20260809 16:34:55.735402 7700 raft_consensus.cc:2229] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:55.735507 7700 raft_consensus.cc:2258] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:55.737668 7702 tablet_replica.cc:330] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: stopping tablet replica
I20260809 16:34:55.737767 7702 raft_consensus.cc:2229] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:34:55.737854 7702 raft_consensus.cc:2258] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:34:55.740265 7700 ts_tablet_manager.cc:1905] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:55.742055 7700 ts_tablet_manager.cc:1918] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:55.742147 7700 log.cc:1196] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/c88046c66e8f491197c26920104f6858
I20260809 16:34:55.742163 7702 ts_tablet_manager.cc:1905] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:34:55.742513 7700 ts_tablet_manager.cc:1939] T c88046c66e8f491197c26920104f6858 P c4622e86e5d046a0a1add23eeaf8b793: Deleting consensus metadata
I20260809 16:34:55.743120 6902 catalog_manager.cc:4841] TS c4622e86e5d046a0a1add23eeaf8b793 (127.31.15.195:34769): tablet c88046c66e8f491197c26920104f6858 (table test-table [id=f73df1bce00f422ab4e60c9727d62c07]) successfully deleted
I20260809 16:34:55.743885 7702 ts_tablet_manager.cc:1918] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:34:55.743948 7702 log.cc:1196] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/c88046c66e8f491197c26920104f6858
I20260809 16:34:55.744163 7702 ts_tablet_manager.cc:1939] T c88046c66e8f491197c26920104f6858 P ca319b4bcc3a460c89819c20fdfb1543: Deleting consensus metadata
I20260809 16:34:55.744936 6898 catalog_manager.cc:4841] TS ca319b4bcc3a460c89819c20fdfb1543 (127.31.15.194:38191): tablet c88046c66e8f491197c26920104f6858 (table test-table [id=f73df1bce00f422ab4e60c9727d62c07]) successfully deleted
W20260809 16:34:55.749683 6898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:38313 (error 108)
I20260809 16:34:55.750140 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.750023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:38313 (local address 127.0.0.1:33400)
0809 16:34:55.750068 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.750111 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:38313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:34:55.750172 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 7226
W20260809 16:34:55.750259 6898 catalog_manager.cc:4584] TS e4910fba32354262bcb64ac66d324215 (127.31.15.193:38313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c88046c66e8f491197c26920104f6858: Network error: Client connection negotiation failed: client connection to 127.31.15.193:38313: connect: Connection refused (error 111)
I20260809 16:34:55.758409 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 7446
I20260809 16:34:55.767060 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 6876
2026-08-09T16:34:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (696 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 16:34:55.790555 31807 mini_oidc.cc:152] Starting JWKS server
I20260809 16:34:55.791688 31807 webserver.cc:460] Webserver started at https://127.0.0.1:35407/ using document root <none> and password file <none>
I20260809 16:34:55.791775 31807 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:34:55.791929 31807 webserver.cc:460] Webserver started at http://127.0.0.1:35551/ using document root <none> and password file <none>
2026-08-09T16:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:55Z Disabled control of system clock
I20260809 16:34:55.800323 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38227
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:37989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38227
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35407/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/testchainca.cert with env {}
W20260809 16:34:55.879930 7715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:55.880125 7715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:55.880147 7715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:55.881237 7715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:55.881270 7715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:55.881282 7715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:55.881292 7715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:55.881304 7715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:34:55.881314 7715 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:34:55.881330 7715 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35407/jwks.json
I20260809 16:34:55.882802 7715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35407/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:55.882989 7715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:55.883143 7715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:55.887696 7722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.887687 7715 server_base.cc:1034] running on GCE node
W20260809 16:34:55.887666 7720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:55.887671 7727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:55.888145 7715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:55.888410 7715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:55.889569 7715 hybrid_clock.cc:648] HybridClock initialized: now 1786293295889555 us; error 39 us; skew 500 ppm
I20260809 16:34:55.890690 7715 webserver.cc:460] Webserver started at http://127.31.15.254:38301/ using document root <none> and password file <none>
I20260809 16:34:55.890892 7715 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:55.890935 7715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:55.891052 7715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:55.892050 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "f566eca01b35454690c55e45b021f1ef"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "9361564c868c3c3835ea7b20ff986830"
server_key_iv: "cfe13d375ab3e376ceed0217f289fbd0"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.892380 7715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "f566eca01b35454690c55e45b021f1ef"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "9361564c868c3c3835ea7b20ff986830"
server_key_iv: "cfe13d375ab3e376ceed0217f289fbd0"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.893601 7715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:34:55.894237 7736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:55.894389 7715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:34:55.894471 7715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "f566eca01b35454690c55e45b021f1ef"
format_stamp: "Formatted at 2026-08-09 16:34:55 on dist-test-slave-21b9"
server_key: "9361564c868c3c3835ea7b20ff986830"
server_key_iv: "cfe13d375ab3e376ceed0217f289fbd0"
server_key_version: "encryptionkey@0"
I20260809 16:34:55.894536 7715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:55.902805 7715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:55.903060 7715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:55.903175 7715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:55.906803 7715 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38227
I20260809 16:34:55.906833 7818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38227 every 8 connection(s)
I20260809 16:34:55.907114 7715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:55.908097 7819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:55.910254 7819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef: Bootstrap starting.
I20260809 16:34:55.910799 7819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:55.911042 7819 log.cc:824] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:55.911685 7819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef: No bootstrap required, opened a new log
I20260809 16:34:55.913172 7819 raft_consensus.cc:348] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } }
I20260809 16:34:55.913301 7819 raft_consensus.cc:374] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:55.913331 7819 raft_consensus.cc:729] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f566eca01b35454690c55e45b021f1ef, State: Initialized, Role: FOLLOWER
I20260809 16:34:55.913430 7819 consensus_queue.cc:260] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } }
I20260809 16:34:55.913478 7819 raft_consensus.cc:388] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:55.913511 7819 raft_consensus.cc:482] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:55.913542 7819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:55.913992 7819 raft_consensus.cc:504] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } }
I20260809 16:34:55.914075 7819 leader_election.cc:302] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f566eca01b35454690c55e45b021f1ef; no voters:
I20260809 16:34:55.914203 7819 leader_election.cc:288] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:55.914292 7822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:55.914439 7819 sys_catalog.cc:564] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:55.914449 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 7715
I20260809 16:34:55.914448 7822 raft_consensus.cc:686] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [term 1 LEADER]: Becoming Leader. State: Replica: f566eca01b35454690c55e45b021f1ef, State: Running, Role: LEADER
I20260809 16:34:55.914551 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:55.914662 7822 consensus_queue.cc:237] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } }
I20260809 16:34:55.914762 31807 external_mini_cluster.cc:1428] Setting key b94b7c66aca616121fc0510ad5b2421a
I20260809 16:34:55.915369 7824 sys_catalog.cc:455] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader f566eca01b35454690c55e45b021f1ef. Latest consensus state: current_term: 1 leader_uuid: "f566eca01b35454690c55e45b021f1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } } }
I20260809 16:34:55.915540 7824 sys_catalog.cc:458] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:55.915799 7844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:55.915541 7823 sys_catalog.cc:455] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f566eca01b35454690c55e45b021f1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f566eca01b35454690c55e45b021f1ef" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38227 } } }
I20260809 16:34:55.915889 7823 sys_catalog.cc:458] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:55.916409 7844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:55.917876 7844 catalog_manager.cc:1347] Generated new cluster ID: 34b3943cd72645328400827bac3842b9
I20260809 16:34:55.917932 7844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:55.921239 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.915195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.31.15.254:38227)
0809 16:34:55.915646 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.915656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.915964 (+ 308us) server_negotiation.cc:407] Connection header received
0809 16:34:55.916196 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.916202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.916265 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.916375 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:55.916713 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.917766 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.918862 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.919522 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.919621 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.920477 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:55.920480 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:55.920481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:55.920668 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:55.920704 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:55.920717 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:55.920764 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:55.920953 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:34:55.921037 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":95,"threads_started":1}
I20260809 16:34:55.922267 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:55.930019 7844 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:55.930707 7844 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:55.945264 7844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f566eca01b35454690c55e45b021f1ef: Generated new TSK 0
I20260809 16:34:55.945461 7844 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:34:55.972261 7858 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 16:34:55.972388 7858 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 16:34:55.972834 7857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:55.958047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38227 (local address 127.0.0.1:42220)
0809 16:34:55.958219 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:55.958228 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:55.958290 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:55.958454 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:55.958455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:55.958457 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:34:55.958525 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.958528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.959138 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.959139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.959779 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:55.959782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.960104 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.960105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:55.960165 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.960186 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:34:55.972484 (+ 12298us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:34:55.972583 (+ 99us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.15.254:38227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260809 16:34:55.973109 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:55.958097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.31.15.254:38227)
0809 16:34:55.958152 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:55.958154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:55.958278 (+ 124us) server_negotiation.cc:407] Connection header received
0809 16:34:55.958372 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:55.958373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:55.958386 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:55.958441 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:34:55.958585 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.959054 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.959852 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:55.960032 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:55.960086 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:55.960292 (+ 206us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:34:55.972345 (+ 12053us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 16:34:55.973046 (+ 701us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 16:34:55.973241 7834 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42220: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.31.15.254:38227: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35407/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 16:34:55.973444 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 7715
2026-08-09T16:34:56Z chronyd exiting
I20260809 16:34:56.022001 31807 test_util.cc:175] -----------------------------------------------
I20260809 16:34:56.022089 31807 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786293272397626-31807-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 16:34:56.022850 31807 mini_oidc.cc:152] Starting JWKS server
I20260809 16:34:56.023578 31807 webserver.cc:460] Webserver started at https://127.0.0.1:38659/ using document root <none> and password file <none>
I20260809 16:34:56.023623 31807 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:34:56.023759 31807 webserver.cc:460] Webserver started at http://127.0.0.1:34471/ using document root <none> and password file <none>
2026-08-09T16:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:56Z Disabled control of system clock
I20260809 16:34:56.032289 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38231
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38231
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38659/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/testchainca.cert with env {}
W20260809 16:34:56.112166 7867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:56.112362 7867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:56.112383 7867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:56.113461 7867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:56.113494 7867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:56.113504 7867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:56.113515 7867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:56.113533 7867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:34:56.113551 7867 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:34:56.113561 7867 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38659/jwks.json
I20260809 16:34:56.115010 7867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38659/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:56.115206 7867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:56.115392 7867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:56.119896 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:56.119925 7867 server_base.cc:1034] running on GCE node
W20260809 16:34:56.119966 7878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:56.119881 7879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:56.120438 7867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:56.120719 7867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:56.121861 7867 hybrid_clock.cc:648] HybridClock initialized: now 1786293296121850 us; error 30 us; skew 500 ppm
I20260809 16:34:56.122968 7867 webserver.cc:460] Webserver started at http://127.31.15.254:36431/ using document root <none> and password file <none>
I20260809 16:34:56.123190 7867 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:56.123238 7867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:56.123389 7867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:56.124361 7867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "2d1ee8ae537a44029fa931302bb5466b"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "401baa662ab5f66f5f073659c6f8ac63"
server_key_iv: "55854ca3c516312fddb6fca4618f6a6c"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.124696 7867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "2d1ee8ae537a44029fa931302bb5466b"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "401baa662ab5f66f5f073659c6f8ac63"
server_key_iv: "55854ca3c516312fddb6fca4618f6a6c"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.125859 7867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:34:56.126494 7894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:56.126629 7867 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:56.126711 7867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "2d1ee8ae537a44029fa931302bb5466b"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "401baa662ab5f66f5f073659c6f8ac63"
server_key_iv: "55854ca3c516312fddb6fca4618f6a6c"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.126775 7867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:56.139550 7867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:56.139796 7867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:56.139912 7867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:56.143735 7867 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38231
I20260809 16:34:56.143766 7978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38231 every 8 connection(s)
I20260809 16:34:56.144068 7867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:56.144698 7979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:56.146255 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 7867
I20260809 16:34:56.146343 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:56.146519 31807 external_mini_cluster.cc:1428] Setting key 6a31804c009fdc45752d1c73ecd28649
I20260809 16:34:56.147511 7979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b: Bootstrap starting.
I20260809 16:34:56.148137 7979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:56.148399 7979 log.cc:824] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:56.149061 7979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b: No bootstrap required, opened a new log
I20260809 16:34:56.151249 7979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } }
I20260809 16:34:56.151429 7979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:56.151466 7979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1ee8ae537a44029fa931302bb5466b, State: Initialized, Role: FOLLOWER
I20260809 16:34:56.151595 7979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } }
I20260809 16:34:56.151686 7979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:56.151719 7979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:56.151774 7979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:56.151897 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.146966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.31.15.254:38231)
0809 16:34:56.147145 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.147152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.147189 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:34:56.147632 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.147641 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.147705 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.147816 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:56.148043 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.148977 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.150027 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.150431 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.150517 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.151131 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:56.151135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:56.151135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:56.151355 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:56.151388 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:56.151399 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:56.151455 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:56.151578 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:34:56.151654 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
I20260809 16:34:56.152369 7979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } }
I20260809 16:34:56.152474 7979 leader_election.cc:302] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d1ee8ae537a44029fa931302bb5466b; no voters:
I20260809 16:34:56.152630 7979 leader_election.cc:288] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:56.152741 7988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:56.152899 7979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:56.152909 7988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [term 1 LEADER]: Becoming Leader. State: Replica: 2d1ee8ae537a44029fa931302bb5466b, State: Running, Role: LEADER
I20260809 16:34:56.153071 7988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } }
I20260809 16:34:56.153378 7988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d1ee8ae537a44029fa931302bb5466b. Latest consensus state: current_term: 1 leader_uuid: "2d1ee8ae537a44029fa931302bb5466b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } } }
I20260809 16:34:56.153513 7988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.153469 7992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d1ee8ae537a44029fa931302bb5466b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1ee8ae537a44029fa931302bb5466b" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38231 } } }
I20260809 16:34:56.153669 7992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.153880 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:56.154052 8006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:56.154573 8006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:56.156032 8006 catalog_manager.cc:1347] Generated new cluster ID: d9059e8f4ff94c329c75d4fc580aff6e
I20260809 16:34:56.156088 8006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:56.163726 8006 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:56.164377 8006 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:56.178483 8006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d1ee8ae537a44029fa931302bb5466b: Generated new TSK 0
I20260809 16:34:56.178658 8006 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 16:34:56.197247 8031 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 16:34:56.197309 8031 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 16:34:56.197640 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.183078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.31.15.254:38231)
0809 16:34:56.183133 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.183134 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.183199 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:34:56.183252 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.183253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.183286 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.183354 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:34:56.183468 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.184235 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.184963 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.185284 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.185348 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.185461 (+ 113us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:34:56.197432 (+ 11971us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38659/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:34:56.197569 (+ 137us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38659/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:34:56.197690 8030 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:56.183009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38231 (local address 127.0.0.1:37054)
0809 16:34:56.183128 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:56.183138 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:56.183193 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:56.183350 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:56.183351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:56.183354 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:34:56.183412 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.183414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.184319 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.184319 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.184892 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.184894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.185346 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.185346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.185381 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.185385 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:34:56.197528 (+ 12143us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38659/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 16:34:56.197620 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.15.254:38231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38659/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
W20260809 16:34:56.197697 7982 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38659/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 16:34:56.197999 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 7867
2026-08-09T16:34:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (226 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 16:34:56.249591 31807 mini_oidc.cc:152] Starting JWKS server
I20260809 16:34:56.250265 31807 webserver.cc:460] Webserver started at https://127.0.0.1:36383/ using document root <none> and password file <none>
I20260809 16:34:56.250310 31807 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 16:34:56.250451 31807 webserver.cc:460] Webserver started at http://127.0.0.1:38585/ using document root <none> and password file <none>
2026-08-09T16:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:56Z Disabled control of system clock
I20260809 16:34:56.258653 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36719
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:39785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:36719
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36383/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:34:56.345126 8039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:56.345387 8039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:56.345418 8039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:56.346563 8039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:56.346602 8039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:56.346619 8039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:56.346634 8039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:56.346650 8039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:34:56.346673 8039 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:34:56.346695 8039 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36383/jwks.json
I20260809 16:34:56.348165 8039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:36719
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36383/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:56.348405 8039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:56.348714 8039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:56.353591 8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:56.353713 8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:56.353614 8054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:56.355327 8039 server_base.cc:1034] running on GCE node
I20260809 16:34:56.355612 8039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:56.355952 8039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:56.357115 8039 hybrid_clock.cc:648] HybridClock initialized: now 1786293296357098 us; error 34 us; skew 500 ppm
I20260809 16:34:56.358739 8039 webserver.cc:460] Webserver started at http://127.31.15.254:45073/ using document root <none> and password file <none>
I20260809 16:34:56.359017 8039 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:56.359087 8039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:56.359232 8039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:56.360736 8039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "34c16d8a9fec4133b603d7f9dc50eda3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "63db7e3cedd6fbdf04b427375b58744b"
server_key_iv: "78cb49063a4c06d1864e95bed9ab8e1e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.361166 8039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "34c16d8a9fec4133b603d7f9dc50eda3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "63db7e3cedd6fbdf04b427375b58744b"
server_key_iv: "78cb49063a4c06d1864e95bed9ab8e1e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.362412 8039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:56.363045 8064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:56.363195 8039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:56.363301 8039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "34c16d8a9fec4133b603d7f9dc50eda3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "63db7e3cedd6fbdf04b427375b58744b"
server_key_iv: "78cb49063a4c06d1864e95bed9ab8e1e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.363363 8039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:56.386307 8039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:56.386586 8039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:56.386688 8039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:56.390511 8039 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:36719
I20260809 16:34:56.390594 8148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:36719 every 8 connection(s)
I20260809 16:34:56.390874 8039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:56.391597 8149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:56.392726 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8039
I20260809 16:34:56.392833 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:56.393014 31807 external_mini_cluster.cc:1428] Setting key 49f15416c7fcd1f52e9e0d1d71725e61
I20260809 16:34:56.394975 8149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3: Bootstrap starting.
I20260809 16:34:56.395643 8149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:56.395893 8149 log.cc:824] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:56.396572 8149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3: No bootstrap required, opened a new log
I20260809 16:34:56.397924 8149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } }
I20260809 16:34:56.398056 8149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:56.398089 8149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c16d8a9fec4133b603d7f9dc50eda3, State: Initialized, Role: FOLLOWER
I20260809 16:34:56.398213 8149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } }
I20260809 16:34:56.398283 8149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:56.398325 8149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:56.398371 8149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:56.398702 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.393533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.31.15.254:36719)
0809 16:34:56.393745 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.393755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.393784 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:34:56.394215 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.394224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.394286 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.394409 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:56.394645 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.395594 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.396558 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.397002 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.397114 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.397986 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:56.397999 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:56.397999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:56.398171 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:56.398213 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:56.398227 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:56.398285 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:56.398416 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:34:56.398503 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260809 16:34:56.398999 8149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } }
I20260809 16:34:56.399102 8149 leader_election.cc:302] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34c16d8a9fec4133b603d7f9dc50eda3; no voters:
I20260809 16:34:56.399247 8149 leader_election.cc:288] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:56.399322 8157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:56.399479 8157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [term 1 LEADER]: Becoming Leader. State: Replica: 34c16d8a9fec4133b603d7f9dc50eda3, State: Running, Role: LEADER
I20260809 16:34:56.399531 8149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:56.399662 8157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } }
I20260809 16:34:56.399856 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:56.400823 8159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34c16d8a9fec4133b603d7f9dc50eda3. Latest consensus state: current_term: 1 leader_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } } }
I20260809 16:34:56.400828 8160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34c16d8a9fec4133b603d7f9dc50eda3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36719 } } }
I20260809 16:34:56.400926 8159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.400942 8160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.401296 8172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:56.401755 8172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:56.403534 8172 catalog_manager.cc:1347] Generated new cluster ID: 2feb021f9dc448ec8d30bf7111e69464
I20260809 16:34:56.403585 8172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:56.411695 8172 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:56.412379 8172 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:56.418725 8172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34c16d8a9fec4133b603d7f9dc50eda3: Generated new TSK 0
I20260809 16:34:56.418885 8172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:56.432612 8152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.429162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51718 (local address 127.31.15.254:36719)
0809 16:34:56.429207 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.429209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.429289 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:34:56.429364 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.429364 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.429374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.429430 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:34:56.429554 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.430041 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.431125 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.431357 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.431403 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.431577 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:34:56.432405 (+ 828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36383/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:34:56.432535 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51718: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36383/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:34:56.432667 8152 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51718: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36383/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 16:34:56.432698 8192 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:56.429068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36719 (local address 127.0.0.1:51718)
0809 16:34:56.429223 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:56.429234 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:56.429299 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:56.429440 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:56.429441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:56.429442 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:34:56.429504 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.429506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.430100 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.430101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.431056 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.431059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.431419 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.431420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.431486 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.431492 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:34:56.432498 (+ 1006us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36383/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 16:34:56.432602 (+ 104us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.15.254:36719: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36383/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 16:34:56.432899 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8039
E20260809 16:34:56.446580 8193 webserver.cc:533] Webserver: sslize failed:
E20260809 16:34:56.446691 8193 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T16:34:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (230 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T16:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:56Z Disabled control of system clock
I20260809 16:34:56.487914 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40907
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40907
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/testchainca.cert with env {}
W20260809 16:34:56.569119 8198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:56.569332 8198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:56.569355 8198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:56.570499 8198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:56.570541 8198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:56.570554 8198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:56.570564 8198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:56.570575 8198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 16:34:56.570600 8198 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 16:34:56.570617 8198 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 16:34:56.572150 8198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:56.572383 8198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:56.572582 8198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:56.577325 8205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:56.577335 8209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:56.577366 8206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:56.577312 8198 server_base.cc:1034] running on GCE node
I20260809 16:34:56.577672 8198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:56.577907 8198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:56.579035 8198 hybrid_clock.cc:648] HybridClock initialized: now 1786293296579018 us; error 34 us; skew 500 ppm
I20260809 16:34:56.580170 8198 webserver.cc:460] Webserver started at http://127.31.15.254:33279/ using document root <none> and password file <none>
I20260809 16:34:56.580395 8198 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:56.580456 8198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:56.580580 8198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:56.581553 8198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "089281ae6adc4a8c85a32966f0effae3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "dc54da8e1c339cfe4efc0af9e9737f08"
server_key_iv: "a0941adcaad909a3f96cce0939d7466e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.581862 8198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "089281ae6adc4a8c85a32966f0effae3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "dc54da8e1c339cfe4efc0af9e9737f08"
server_key_iv: "a0941adcaad909a3f96cce0939d7466e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.583063 8198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:34:56.583819 8224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:56.583945 8198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:56.584033 8198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "089281ae6adc4a8c85a32966f0effae3"
format_stamp: "Formatted at 2026-08-09 16:34:56 on dist-test-slave-21b9"
server_key: "dc54da8e1c339cfe4efc0af9e9737f08"
server_key_iv: "a0941adcaad909a3f96cce0939d7466e"
server_key_version: "encryptionkey@0"
I20260809 16:34:56.584102 8198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:56.604278 8198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:56.604527 8198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:56.604643 8198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:56.608289 8198 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40907
I20260809 16:34:56.608330 8316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40907 every 8 connection(s)
I20260809 16:34:56.608599 8198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:56.609525 8318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:56.611728 8318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3: Bootstrap starting.
I20260809 16:34:56.612120 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8198
I20260809 16:34:56.612226 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:56.612344 8318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:56.612406 31807 external_mini_cluster.cc:1428] Setting key f67ef0a43619b6d464d620d3c3595522
I20260809 16:34:56.612614 8318 log.cc:824] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:56.613366 8318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3: No bootstrap required, opened a new log
I20260809 16:34:56.616619 8318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } }
I20260809 16:34:56.616779 8318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:56.616816 8318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089281ae6adc4a8c85a32966f0effae3, State: Initialized, Role: FOLLOWER
I20260809 16:34:56.616945 8318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } }
I20260809 16:34:56.617029 8318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:56.617059 8318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:56.617098 8318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:56.617756 8318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } }
I20260809 16:34:56.617867 8318 leader_election.cc:302] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089281ae6adc4a8c85a32966f0effae3; no voters:
I20260809 16:34:56.618031 8318 leader_election.cc:288] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:56.618073 8329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:56.618245 8329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [term 1 LEADER]: Becoming Leader. State: Replica: 089281ae6adc4a8c85a32966f0effae3, State: Running, Role: LEADER
I20260809 16:34:56.618314 8318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:56.618337 8329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } }
I20260809 16:34:56.618682 8329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089281ae6adc4a8c85a32966f0effae3. Latest consensus state: current_term: 1 leader_uuid: "089281ae6adc4a8c85a32966f0effae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } } }
I20260809 16:34:56.618747 8329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.619608 8330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089281ae6adc4a8c85a32966f0effae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089281ae6adc4a8c85a32966f0effae3" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40907 } } }
I20260809 16:34:56.619665 8344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:56.619658 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.612860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.31.15.254:40907)
0809 16:34:56.613233 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.613242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.613311 (+ 69us) server_negotiation.cc:407] Connection header received
0809 16:34:56.614432 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.614441 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.614517 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.614664 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:56.614852 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.616221 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.617355 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.618048 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.618502 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.618890 (+ 388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:56.618894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:56.618894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:34:56.619089 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:56.619130 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:34:56.619144 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:56.619244 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:56.619379 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:34:56.619463 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":81,"threads_started":1}
I20260809 16:34:56.619750 8330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:56.620428 8344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:56.620728 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:34:56.622031 8344 catalog_manager.cc:1347] Generated new cluster ID: 9914645c0d554a1bab57700223943e80
I20260809 16:34:56.622083 8344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:34:56.627646 8344 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:56.628109 8344 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:56.630393 8344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 089281ae6adc4a8c85a32966f0effae3: Generated new TSK 0
I20260809 16:34:56.630533 8344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:34:56.710404 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:56.634816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.31.15.254:40907)
0809 16:34:56.634859 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:56.634860 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:56.634955 (+ 95us) server_negotiation.cc:407] Connection header received
0809 16:34:56.635031 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:56.635031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:56.635042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:56.635096 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0809 16:34:56.635213 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.635720 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.636476 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.636670 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.636722 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.636878 (+ 156us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 16:34:56.710147 (+ 73269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:34:56.710326 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50128: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:34:56.710489 8327 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50128: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 16:34:56.710518 8370 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:56.634763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40907 (local address 127.0.0.1:50128)
0809 16:34:56.634895 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:56.634905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:56.634963 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:56.635100 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:56.635101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:56.635103 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 16:34:56.635158 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.635160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.635809 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.635810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.636405 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:56.636407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:56.636737 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:56.636738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:56.636796 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:56.636801 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 16:34:56.710295 (+ 73494us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 16:34:56.710423 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.31.15.254:40907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260809 16:34:56.710927 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8198
2026-08-09T16:34:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (256 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:56 dist-test-slave-21b9 krb5kdc[8388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:56 dist-test-slave-21b9 krb5kdc[8388](info): set up 2 sockets
Aug 09 16:34:56 dist-test-slave-21b9 krb5kdc[8388](info): commencing operation
krb5kdc: starting...
W20260809 16:34:58.042498 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:34:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:34:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:34:58.159122 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46193
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:58.238600 8411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:58.238801 8411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:58.238821 8411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:58.240006 8411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:34:58.240049 8411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:58.240062 8411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:58.240073 8411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:34:58.240084 8411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:34:58.241533 8411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:58.241767 8411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:58.241976 8411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:58.246505 8425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.246512 8428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.246512 8424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:58.246703 8411 server_base.cc:1034] running on GCE node
I20260809 16:34:58.246996 8411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:58.247238 8411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:58.248394 8411 hybrid_clock.cc:648] HybridClock initialized: now 1786293298248376 us; error 37 us; skew 500 ppm
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.250754 8411 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:34:58.251154 8411 webserver.cc:460] Webserver started at http://127.31.15.254:42049/ using document root <none> and password file <none>
I20260809 16:34:58.251397 8411 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:58.251448 8411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:58.251585 8411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:58.252578 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "79070d087a0d444bb84f69afd36412b5"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "51357cb89bc487e3db0cf5f78e05759b"
server_key_iv: "06d002271c85a25de3d25558a2c0d2b2"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.252918 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "79070d087a0d444bb84f69afd36412b5"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "51357cb89bc487e3db0cf5f78e05759b"
server_key_iv: "06d002271c85a25de3d25558a2c0d2b2"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.254132 8411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:34:58.254778 8442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.254927 8411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:58.255008 8411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "79070d087a0d444bb84f69afd36412b5"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "51357cb89bc487e3db0cf5f78e05759b"
server_key_iv: "06d002271c85a25de3d25558a2c0d2b2"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.255074 8411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:58.268648 8411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:58.270311 8411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:58.270438 8411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:58.274309 8411 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:46193
I20260809 16:34:58.274343 8523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:46193 every 8 connection(s)
I20260809 16:34:58.274672 8411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:34:58.275547 8524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:34:58.277784 8524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5: Bootstrap starting.
I20260809 16:34:58.278342 8524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5: Neither blocks nor log segments found. Creating new log.
I20260809 16:34:58.278617 8524 log.cc:824] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:34:58.279237 8524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5: No bootstrap required, opened a new log
I20260809 16:34:58.280836 8524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } }
I20260809 16:34:58.280970 8524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:34:58.281003 8524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79070d087a0d444bb84f69afd36412b5, State: Initialized, Role: FOLLOWER
I20260809 16:34:58.281138 8524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } }
I20260809 16:34:58.281198 8524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:34:58.281219 8524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:34:58.281265 8524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:34:58.281754 8524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } }
I20260809 16:34:58.281839 8524 leader_election.cc:302] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79070d087a0d444bb84f69afd36412b5; no voters:
I20260809 16:34:58.281978 8524 leader_election.cc:288] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:34:58.282040 8528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:34:58.282209 8524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:34:58.282387 8528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [term 1 LEADER]: Becoming Leader. State: Replica: 79070d087a0d444bb84f69afd36412b5, State: Running, Role: LEADER
I20260809 16:34:58.282485 8528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } }
I20260809 16:34:58.282833 8530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79070d087a0d444bb84f69afd36412b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } } }
I20260809 16:34:58.282840 8531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79070d087a0d444bb84f69afd36412b5. Latest consensus state: current_term: 1 leader_uuid: "79070d087a0d444bb84f69afd36412b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79070d087a0d444bb84f69afd36412b5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46193 } } }
I20260809 16:34:58.282982 8530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:58.282990 8531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:34:58.283432 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8411
I20260809 16:34:58.283509 8546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:34:58.283540 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:34:58.283744 31807 external_mini_cluster.cc:1428] Setting key 7b1f5692b1eeadc9f126dfdda42f5fb1
I20260809 16:34:58.283989 8546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:34:58.285983 8546 catalog_manager.cc:1347] Generated new cluster ID: 884ba782789d44c9ba5f86754900af9e
I20260809 16:34:58.286041 8546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:58.293480 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.284349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.31.15.254:46193)
0809 16:34:58.284525 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.284534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.284557 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:34:58.284675 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.284682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.284735 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.284869 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.285645 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.286620 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.287407 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.287812 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.287917 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.290065 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.290070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.290070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.290082 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.292105 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.292425 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.292429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.292429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.292510 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.292681 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.292682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.292682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.292909 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.293055 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.293155 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.293286 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
I20260809 16:34:58.297904 8546 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:34:58.298413 8546 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:34:58.302726 8546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79070d087a0d444bb84f69afd36412b5: Generated new TSK 0
I20260809 16:34:58.302886 8546 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:34:58.332209 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:58.420576 8573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:58.420768 8573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:58.420790 8573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:58.421862 8573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:58.421895 8573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:58.421921 8573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:34:58.423331 8573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:58.423516 8573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:58.423682 8573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:58.428231 8581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.428237 8584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.428237 8580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:58.428473 8573 server_base.cc:1034] running on GCE node
I20260809 16:34:58.428664 8573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:58.428845 8573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:58.429973 8573 hybrid_clock.cc:648] HybridClock initialized: now 1786293298429959 us; error 30 us; skew 500 ppm
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.432382 8573 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:34:58.432801 8573 webserver.cc:460] Webserver started at http://127.31.15.193:43625/ using document root <none> and password file <none>
I20260809 16:34:58.432992 8573 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:58.433038 8573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:58.433161 8573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:58.434129 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "8e9da0d63e3f43888b6f6fe9e377e391"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "70a4d519e9b1253a4e2dcc2f11989006"
server_key_iv: "061c729d16d998891d54bb4fabd293e4"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.434442 8573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "8e9da0d63e3f43888b6f6fe9e377e391"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "70a4d519e9b1253a4e2dcc2f11989006"
server_key_iv: "061c729d16d998891d54bb4fabd293e4"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.435678 8573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:58.436316 8592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.436465 8573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:34:58.436547 8573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "8e9da0d63e3f43888b6f6fe9e377e391"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "70a4d519e9b1253a4e2dcc2f11989006"
server_key_iv: "061c729d16d998891d54bb4fabd293e4"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.436604 8573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:58.449378 8573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:58.451112 8573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:58.451239 8573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:58.451486 8573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:58.451772 8573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:58.451805 8573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.451833 8573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:58.451856 8573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.457347 8573 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45857
I20260809 16:34:58.457396 8765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45857 every 8 connection(s)
I20260809 16:34:58.457710 8573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:58.465988 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.458377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:60575 (local address 127.31.15.254:46193)
0809 16:34:58.458459 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.458461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.458612 (+ 151us) server_negotiation.cc:407] Connection header received
0809 16:34:58.459553 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.459554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.459564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.459619 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.460160 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.460662 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.461565 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.461760 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.461807 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.463677 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.463678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.463678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.463685 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.464924 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.465221 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.465222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.465222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.465270 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.465459 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.465460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.465461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.465570 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.465620 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.465847 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.465914 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:34:58.466118 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.458340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46193 (local address 127.31.15.193:60575)
0809 16:34:58.458499 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:58.458538 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:58.459467 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:58.459673 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:58.459676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:58.459914 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:58.460096 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.460100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.460724 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.460725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.461503 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.461505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.461814 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.461815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.461905 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.462293 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:58.462301 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:58.463594 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:58.464990 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.464993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.464994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.465173 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.465311 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.465311 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.465311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.465392 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.465629 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:58.465635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:58.465746 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:34:58.465828 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:34:58.465941 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260809 16:34:58.466638 8766 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46193
I20260809 16:34:58.466722 8766 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:58.466868 8766 heartbeater.cc:511] Master 127.31.15.254:46193 requested a full tablet report, sending...
I20260809 16:34:58.467392 8464 ts_manager.cc:194] Registered new tserver with Master: 8e9da0d63e3f43888b6f6fe9e377e391 (127.31.15.193:45857)
I20260809 16:34:58.467489 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8573
I20260809 16:34:58.467595 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:34:58.467788 31807 external_mini_cluster.cc:1428] Setting key 5a8eff33c39b0f106407e6053bb2ba2c
I20260809 16:34:58.468341 8464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:60575
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:34:58.491324 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:58.571949 8780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:58.572147 8780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:58.572180 8780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:58.573300 8780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:58.573340 8780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:58.573372 8780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:34:58.574817 8780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:58.575052 8780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:58.575250 8780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:58.579962 8785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.580008 8786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.579946 8788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:58.580164 8780 server_base.cc:1034] running on GCE node
I20260809 16:34:58.580394 8780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:58.580595 8780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:58.581750 8780 hybrid_clock.cc:648] HybridClock initialized: now 1786293298581729 us; error 37 us; skew 500 ppm
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.584306 8780 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:34:58.584695 8780 webserver.cc:460] Webserver started at http://127.31.15.194:41403/ using document root <none> and password file <none>
I20260809 16:34:58.584885 8780 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:58.584923 8780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:58.585026 8780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:58.586011 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "f114066cdbd847f6af86a3be97a8cc0c"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "cc8cdee8ec1fa452b6a3faebb7ae23df"
server_key_iv: "268b5d2ab492d1bf2224f955fe300529"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.586297 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "f114066cdbd847f6af86a3be97a8cc0c"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "cc8cdee8ec1fa452b6a3faebb7ae23df"
server_key_iv: "268b5d2ab492d1bf2224f955fe300529"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.587495 8780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:34:58.588224 8797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.588403 8780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:34:58.588487 8780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "f114066cdbd847f6af86a3be97a8cc0c"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "cc8cdee8ec1fa452b6a3faebb7ae23df"
server_key_iv: "268b5d2ab492d1bf2224f955fe300529"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.588546 8780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:58.595432 8780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:58.597267 8780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:58.597374 8780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:58.597570 8780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:58.597843 8780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:58.597875 8780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.597898 8780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:58.597919 8780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.603503 8780 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:39271
I20260809 16:34:58.603539 8975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:39271 every 8 connection(s)
I20260809 16:34:58.603818 8780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:34:58.606060 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8780
I20260809 16:34:58.606160 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:34:58.606320 31807 external_mini_cluster.cc:1428] Setting key e6a6f4c2c6358e789c89d0c19d8409f5
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:58.613420 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.604463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:54311 (local address 127.31.15.254:46193)
0809 16:34:58.604519 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.604521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.604704 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:34:58.605576 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.605576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.605585 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.605638 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.606226 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.606811 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.607865 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.608130 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.608202 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.610357 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.610358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.610358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.610367 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.611981 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.612465 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.612466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.612466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.612557 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.612841 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.612841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.612841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.612944 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.612989 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.613287 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.613355 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:34:58.613660 8979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.604433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46193 (local address 127.31.15.194:54311)
0809 16:34:58.604591 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:58.604632 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:58.605497 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:58.605689 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:58.605692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:58.605941 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:58.606123 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.606127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.606921 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.606921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.607781 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.607784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.608201 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.608202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.608298 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.608769 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:58.608779 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:58.610270 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:58.612055 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.612062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.612064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.612312 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.612629 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.612630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.612631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.612776 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.612998 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:58.613003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:58.613161 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 16:34:58.613278 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 16:34:58.613439 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260809 16:34:58.614181 8977 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46193
I20260809 16:34:58.614284 8977 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:58.614473 8977 heartbeater.cc:511] Master 127.31.15.254:46193 requested a full tablet report, sending...
I20260809 16:34:58.614853 8464 ts_manager.cc:194] Registered new tserver with Master: f114066cdbd847f6af86a3be97a8cc0c (127.31.15.194:39271)
I20260809 16:34:58.615378 8464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:54311
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:34:58.631919 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:42937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:34:58.711750 8984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:34:58.711941 8984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:34:58.711961 8984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:34:58.713024 8984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:34:58.713055 8984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:34:58.713080 8984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:34:58.714476 8984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:42937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:34:58.714665 8984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:34:58.714835 8984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:34:58.719478 8998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.719472 8996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:34:58.719471 8995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:34:58.719789 8984 server_base.cc:1034] running on GCE node
I20260809 16:34:58.720003 8984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:34:58.720213 8984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:34:58.721346 8984 hybrid_clock.cc:648] HybridClock initialized: now 1786293298721332 us; error 30 us; skew 500 ppm
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.723685 8984 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:34:58.724123 8984 webserver.cc:460] Webserver started at http://127.31.15.195:39917/ using document root <none> and password file <none>
I20260809 16:34:58.724347 8984 fs_manager.cc:359] Metadata directory not provided
I20260809 16:34:58.724395 8984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:34:58.724519 8984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:34:58.725476 8984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "49f0d464fcbc43a290b05a7f01a03f7d"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "e9d060bd9dcd5ca37560b082e03d9595"
server_key_iv: "ecdf8ecdfe084d01d2599ca28d638b51"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.725797 8984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "49f0d464fcbc43a290b05a7f01a03f7d"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "e9d060bd9dcd5ca37560b082e03d9595"
server_key_iv: "ecdf8ecdfe084d01d2599ca28d638b51"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.727025 8984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:34:58.727730 9009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.727880 8984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.727968 8984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "49f0d464fcbc43a290b05a7f01a03f7d"
format_stamp: "Formatted at 2026-08-09 16:34:58 on dist-test-slave-21b9"
server_key: "e9d060bd9dcd5ca37560b082e03d9595"
server_key_iv: "ecdf8ecdfe084d01d2599ca28d638b51"
server_key_version: "encryptionkey@0"
I20260809 16:34:58.728032 8984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:34:58.743785 8984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:34:58.745435 8984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:34:58.745561 8984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:34:58.745756 8984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:34:58.746033 8984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:34:58.746065 8984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.746093 8984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:34:58.746116 8984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:34:58.751463 8984 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:46187
I20260809 16:34:58.751523 9193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:46187 every 8 connection(s)
I20260809 16:34:58.751801 8984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:34:58.756601 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 8984
I20260809 16:34:58.756698 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:34:58.756887 31807 external_mini_cluster.cc:1428] Setting key c3fa4a97b7e776895f4a9aa8ca17bfbf
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:34:58.760231 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.752435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:45793 (local address 127.31.15.254:46193)
0809 16:34:58.752537 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.752540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.752655 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:34:58.753551 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.753553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.753563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.753607 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.754162 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.754677 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.755681 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.755886 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.755938 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.758088 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.758089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.758089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.758095 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.759196 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.759572 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.759572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.759573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.759621 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.759789 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.759790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.759790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.759882 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.759925 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.760114 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.760169 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:34:58.760389 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.752400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46193 (local address 127.31.15.195:45793)
0809 16:34:58.752541 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:34:58.752577 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:34:58.753441 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:34:58.753666 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:34:58.753669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:34:58.753904 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:34:58.754088 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.754092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.754758 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.754759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.755613 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:34:58.755615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.755952 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.755953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:34:58.756043 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.756436 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:34:58.756446 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:34:58.758006 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:34:58.759259 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.759264 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.759285 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.759509 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.759663 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:34:58.759664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:34:58.759664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:34:58.759748 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:34:58.759921 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:34:58.759923 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:34:58.760026 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:34:58.760100 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:34:58.760208 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260809 16:34:58.760746 9194 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46193
I20260809 16:34:58.760835 9194 heartbeater.cc:464] Registering TS with master...
I20260809 16:34:58.761004 9194 heartbeater.cc:511] Master 127.31.15.254:46193 requested a full tablet report, sending...
I20260809 16:34:58.761330 8464 ts_manager.cc:194] Registered new tserver with Master: 49f0d464fcbc43a290b05a7f01a03f7d (127.31.15.195:46187)
I20260809 16:34:58.761674 8464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:45793
I20260809 16:34:58.768532 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/test-admin.keytab.
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.784116 31807 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 16:34:58.785698 9205 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:58.784929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50964 (local address 127.31.15.193:45857)
0809 16:34:58.785083 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.785088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.785096 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:58.785120 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.785121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.785134 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.785197 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.785619 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50964: BlockingRecv error: recv got EOF from 127.0.0.1:50964 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
W20260809 16:34:58.785768 31807 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc)
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.786633 31807 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.787461 31807 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:58.794483 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.787927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.31.15.193:45857)
0809 16:34:58.787981 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.787983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.787994 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:58.788055 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.788056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.788067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.788115 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.788474 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.789190 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.789886 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.790206 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.790265 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.792215 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.792223 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.792223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.792229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.793616 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.793895 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.793897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.793897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.793947 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.794067 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.794067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.794068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.794180 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.794268 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.794321 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.794379 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 16:34:58.795982 9205 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:58.795598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.31.15.193:45857)
0809 16:34:58.795633 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.795634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.795663 (+ 29us) server_negotiation.cc:407] Connection header received
0809 16:34:58.795713 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.795714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.795722 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.795754 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.795937 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50978: BlockingRecv error: recv got EOF from 127.0.0.1:50978 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:34:58.796052 31807 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.796866 31807 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.797598 31807 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:58.803547 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.798023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.31.15.193:45857)
0809 16:34:58.798065 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.798066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.798075 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:34:58.798118 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.798119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.798126 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.798165 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.798437 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.798906 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.799585 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.799769 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.799811 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.801467 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.801468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.801468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.801474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.802471 (+ 997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.802855 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.802857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.802857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.802915 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.803091 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.803092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.803093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.803182 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.803257 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.803393 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.803465 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 16:34:58.804708 9205 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:34:58.804335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.31.15.193:45857)
0809 16:34:58.804367 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.804369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.804380 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:34:58.804422 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.804422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.804429 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.804468 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.804668 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50988: BlockingRecv error: recv got EOF from 127.0.0.1:50988 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:34:58.804804 31807 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786293272397626-31807-0/krb5kdc/krb5cc' of type FILE
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.805605 31807 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:34:58.806329 31807 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[8388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:34:58.812126 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:34:58.806824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50994 (local address 127.31.15.193:45857)
0809 16:34:58.806891 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:34:58.806892 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:34:58.806900 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:34:58.806940 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:34:58.806941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:34:58.806948 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:34:58.806988 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:34:58.807400 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.807828 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.808437 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:34:58.808603 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:34:58.808643 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:34:58.810311 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:34:58.810312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:34:58.810312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:34:58.810318 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:34:58.811375 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.811645 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.811646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.811646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.811688 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:34:58.811793 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:34:58.811793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:34:58.811794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:34:58.811864 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:34:58.811920 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:34:58.811992 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:34:58.812050 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:34:58.812682 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8573
I20260809 16:34:58.827124 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8780
I20260809 16:34:58.841259 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8984
I20260809 16:34:58.850133 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 8411
2026-08-09T16:34:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1959 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[9224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[9224](info): set up 2 sockets
Aug 09 16:34:58 dist-test-slave-21b9 krb5kdc[9224](info): commencing operation
krb5kdc: starting...
W20260809 16:35:00.002696 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab.
Entry for principal oryx/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab.
I20260809 16:35:00.119128 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40851
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab
--principal=oryx/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:00.199196 9244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:00.199416 9244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:00.199446 9244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:00.200567 9244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:00.200609 9244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:00.200628 9244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:00.200644 9244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:00.200659 9244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:00.202111 9244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab
--principal=oryx/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:00.202358 9244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:00.202569 9244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:00.207059 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:00.207094 9252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:00.207067 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.207300 9244 server_base.cc:1034] running on GCE node
I20260809 16:35:00.207501 9244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:00.207734 9244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:00.208873 9244 hybrid_clock.cc:648] HybridClock initialized: now 1786293300208858 us; error 30 us; skew 500 ppm
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:00.213800 9244 init.cc:377] Logged in from keytab as oryx/127.31.15.254@KRBTEST.COM (short username oryx)
I20260809 16:35:00.214221 9244 webserver.cc:460] Webserver started at http://127.31.15.254:44633/ using document root <none> and password file <none>
I20260809 16:35:00.214437 9244 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:00.214479 9244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:00.214610 9244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:00.215675 9244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "af165bb537bf4043aa288845e9f93043"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "ef1998cd1ec5e488131ea92711b7f2ca"
server_key_iv: "c5071e415f0ee72019209c315311db8c"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.216010 9244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "af165bb537bf4043aa288845e9f93043"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "ef1998cd1ec5e488131ea92711b7f2ca"
server_key_iv: "c5071e415f0ee72019209c315311db8c"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.217268 9244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:00.217989 9269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.218142 9244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:00.218231 9244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "af165bb537bf4043aa288845e9f93043"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "ef1998cd1ec5e488131ea92711b7f2ca"
server_key_iv: "c5071e415f0ee72019209c315311db8c"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.218291 9244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:00.234030 9244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:00.236035 9244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:00.236182 9244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:00.240564 9244 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40851
I20260809 16:35:00.240614 9358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40851 every 8 connection(s)
I20260809 16:35:00.240936 9244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:00.241914 9359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:00.243624 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 9244
I20260809 16:35:00.243743 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:00.243986 31807 external_mini_cluster.cc:1428] Setting key c533b2e734efcea23934830d3b9dd8e0
I20260809 16:35:00.244629 9359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Bootstrap starting.
I20260809 16:35:00.245213 9359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:00.245499 9359 log.cc:824] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:00.246191 9359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: No bootstrap required, opened a new log
I20260809 16:35:00.247969 9359 raft_consensus.cc:348] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } }
I20260809 16:35:00.248106 9359 raft_consensus.cc:374] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:00.248139 9359 raft_consensus.cc:729] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af165bb537bf4043aa288845e9f93043, State: Initialized, Role: FOLLOWER
I20260809 16:35:00.248265 9359 consensus_queue.cc:260] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } }
I20260809 16:35:00.248349 9359 raft_consensus.cc:388] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:00.248406 9359 raft_consensus.cc:482] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:00.248457 9359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:00.248973 9359 raft_consensus.cc:504] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } }
I20260809 16:35:00.249058 9359 leader_election.cc:302] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af165bb537bf4043aa288845e9f93043; no voters:
I20260809 16:35:00.249225 9359 leader_election.cc:288] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:00.249258 9364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:00.249358 9364 raft_consensus.cc:686] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 LEADER]: Becoming Leader. State: Replica: af165bb537bf4043aa288845e9f93043, State: Running, Role: LEADER
I20260809 16:35:00.249440 9364 consensus_queue.cc:237] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } }
I20260809 16:35:00.249496 9359 sys_catalog.cc:564] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:00.249748 9364 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } }
I20260809 16:35:00.249889 9364 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:00.250087 9364 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af165bb537bf4043aa288845e9f93043. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } }
I20260809 16:35:00.250200 9364 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:00.250600 9371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:00.251169 9371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:00.252851 9371 catalog_manager.cc:1347] Generated new cluster ID: e85e7a17a4e440bca1d5638ce5796862
I20260809 16:35:00.252895 9371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:00.254604 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.244505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.31.15.254:40851)
0809 16:35:00.244680 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:00.244690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:00.244724 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:35:00.245183 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:00.245190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:00.245238 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:00.245376 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:00.246140 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.247033 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.247805 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.248249 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.248349 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.250986 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:00.250991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:00.250992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:00.251005 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:00.252910 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.253327 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.253330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.253330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.253435 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.253643 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.253643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.253644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.253912 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:00.254041 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:00.254231 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 16:35:00.254373 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260809 16:35:00.267732 9371 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:00.268483 9371 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:00.274756 9371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Generated new TSK 0
I20260809 16:35:00.274931 9371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
I20260809 16:35:00.300102 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:00.381520 9402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:00.381717 9402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:00.381739 9402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:00.382843 9402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:00.382893 9402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:00.382920 9402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:00.384336 9402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:00.384549 9402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:00.384760 9402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:00.389398 9416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:00.389436 9412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.389546 9402 server_base.cc:1034] running on GCE node
W20260809 16:35:00.389573 9418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.389847 9402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:00.390080 9402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:00.391294 9402 hybrid_clock.cc:648] HybridClock initialized: now 1786293300391254 us; error 25 us; skew 500 ppm
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:00.393633 9402 init.cc:377] Logged in from keytab as oryx/127.31.15.193@KRBTEST.COM (short username oryx)
I20260809 16:35:00.394063 9402 webserver.cc:460] Webserver started at http://127.31.15.193:37275/ using document root <none> and password file <none>
I20260809 16:35:00.394263 9402 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:00.394311 9402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:00.394433 9402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:00.395471 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "fd4eba99d45d4d799f5b5381e7758c81"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "60f1e6554de84c2470abdab42042d468"
server_key_iv: "fad76f7fa0f425c48a6bdc0c2704af21"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.395789 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "fd4eba99d45d4d799f5b5381e7758c81"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "60f1e6554de84c2470abdab42042d468"
server_key_iv: "fad76f7fa0f425c48a6bdc0c2704af21"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.397017 9402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:00.397634 9431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.397776 9402 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:00.397859 9402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "fd4eba99d45d4d799f5b5381e7758c81"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "60f1e6554de84c2470abdab42042d468"
server_key_iv: "fad76f7fa0f425c48a6bdc0c2704af21"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.397917 9402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:00.416031 9402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:00.417724 9402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:00.417862 9402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:00.418073 9402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:00.418359 9402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:00.418394 9402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.418421 9402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:00.418437 9402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.423907 9402 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45009
I20260809 16:35:00.423966 9610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45009 every 8 connection(s)
I20260809 16:35:00.424247 9402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:35:00.424927 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 9402
I20260809 16:35:00.425041 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:00.425252 31807 external_mini_cluster.cc:1428] Setting key 4adbcc7f67c2660e5a81f09e0a68fe42
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:00.433898 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.424891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39753 (local address 127.31.15.254:40851)
0809 16:35:00.424955 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:00.424958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:00.425140 (+ 182us) server_negotiation.cc:407] Connection header received
0809 16:35:00.425698 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:00.425699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:00.425709 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:00.425768 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:00.426710 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.427414 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.428821 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.429084 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.429182 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.431315 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:00.431317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:00.431317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:00.431325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:00.432658 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.433041 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.433042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.433043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.433096 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.433304 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.433304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.433305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.433425 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:00.433481 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:00.433728 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 16:35:00.433809 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:35:00.434082 9616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.424836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:39753)
0809 16:35:00.425022 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:00.425063 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:00.425603 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:00.425844 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:00.425849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:00.426413 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:00.426619 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.426624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.427534 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.427535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.428721 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.428725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.429186 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.429187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.429288 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.429714 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:00.429722 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:00.431207 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:00.432748 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.432755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.432757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.432974 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.433150 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.433151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.433151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.433250 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.433481 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:00.433487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:00.433616 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 16:35:00.433700 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 16:35:00.433848 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.31.15.194@KRBTEST.COM; defaulting to no policy
I20260809 16:35:00.434540 9611 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:00.434645 9611 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:00.434839 9611 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:00.435374 9293 ts_manager.cc:194] Registered new tserver with Master: fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009)
I20260809 16:35:00.435998 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:39753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
I20260809 16:35:00.451159 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:00.532642 9621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:00.532832 9621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:00.532855 9621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:00.533957 9621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:00.533995 9621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:00.534024 9621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:00.535506 9621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:00.535720 9621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:00.535921 9621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:00.540452 9633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.540544 9621 server_base.cc:1034] running on GCE node
W20260809 16:35:00.540482 9628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:00.540460 9630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.540800 9621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:00.540998 9621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:00.542131 9621 hybrid_clock.cc:648] HybridClock initialized: now 1786293300542122 us; error 31 us; skew 500 ppm
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:00.544658 9621 init.cc:377] Logged in from keytab as oryx/127.31.15.194@KRBTEST.COM (short username oryx)
I20260809 16:35:00.545059 9621 webserver.cc:460] Webserver started at http://127.31.15.194:45543/ using document root <none> and password file <none>
I20260809 16:35:00.545251 9621 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:00.545300 9621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:00.545425 9621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:00.546382 9621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "2af9bdf77e0d472fa940ef53366c1775"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "df7d2900f4d8ef691312ff157df7ce89"
server_key_iv: "e5a9fbdea896516c78dab35fab466c38"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.546705 9621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "2af9bdf77e0d472fa940ef53366c1775"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "df7d2900f4d8ef691312ff157df7ce89"
server_key_iv: "e5a9fbdea896516c78dab35fab466c38"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.547977 9621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:35:00.548660 9649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.548830 9621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:00.548918 9621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "2af9bdf77e0d472fa940ef53366c1775"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "df7d2900f4d8ef691312ff157df7ce89"
server_key_iv: "e5a9fbdea896516c78dab35fab466c38"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.548975 9621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:00.561213 9621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:00.562994 9621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:00.563115 9621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:00.563424 9621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:00.563719 9621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:00.563752 9621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.563781 9621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:00.563805 9621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.569051 9621 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:36171
I20260809 16:35:00.569118 9854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:36171 every 8 connection(s)
I20260809 16:35:00.569396 9621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:00.575839 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 9621
I20260809 16:35:00.575948 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:00.576130 31807 external_mini_cluster.cc:1428] Setting key f557032adef2c5433938d53f57dde4a3
I20260809 16:35:00.578377 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.570071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:47995 (local address 127.31.15.254:40851)
0809 16:35:00.570132 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:00.570134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:00.570315 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:35:00.570860 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:00.570861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:00.570875 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:00.570929 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:00.571916 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.572411 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.573354 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.573551 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.573608 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.575598 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:00.575599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:00.575599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:00.575607 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:00.577001 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.577418 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.577418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.577419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.577469 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.577731 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.577733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.577733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.577853 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:00.577926 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:00.578239 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 16:35:00.578311 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:35:00.578568 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.570029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:47995)
0809 16:35:00.570202 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:00.570241 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:00.570761 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:00.571008 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:00.571013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:00.571621 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:00.571812 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.571817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.572491 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.572492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.573274 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.573276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.573619 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.573620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.573738 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.574136 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:00.574143 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:00.575507 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:00.577076 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.577084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.577086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.577347 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.577517 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.577518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.577518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.577647 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.577927 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:00.577932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:00.578088 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 16:35:00.578195 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 16:35:00.578350 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:35:00.579046 9855 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:00.579131 9855 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:00.579336 9855 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:00.579708 9293 ts_manager.cc:194] Registered new tserver with Master: 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171)
I20260809 16:35:00.580065 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:47995
WARNING: no policy specified for oryx/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
I20260809 16:35:00.601310 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:00.680743 9864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:00.680938 9864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:00.680968 9864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:00.682080 9864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:00.682121 9864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:00.682155 9864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:00.683622 9864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:00.683884 9864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:00.684103 9864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:00.688921 9876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:00.688915 9874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.688962 9864 server_base.cc:1034] running on GCE node
W20260809 16:35:00.688915 9871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:00.689289 9864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:00.689512 9864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:00.690644 9864 hybrid_clock.cc:648] HybridClock initialized: now 1786293300690633 us; error 25 us; skew 500 ppm
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:00.693295 9864 init.cc:377] Logged in from keytab as oryx/127.31.15.195@KRBTEST.COM (short username oryx)
I20260809 16:35:00.693708 9864 webserver.cc:460] Webserver started at http://127.31.15.195:34811/ using document root <none> and password file <none>
I20260809 16:35:00.693948 9864 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:00.694010 9864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:00.694145 9864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:00.695233 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "86a5a108917415b9bc10810d853fe907"
server_key_iv: "c6ec1107d30d82a1118e4ffbea4ecc93"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.695616 9864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "86a5a108917415b9bc10810d853fe907"
server_key_iv: "c6ec1107d30d82a1118e4ffbea4ecc93"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.696839 9864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:00.697503 9886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.697657 9864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:00.697736 9864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "86a5a108917415b9bc10810d853fe907"
server_key_iv: "c6ec1107d30d82a1118e4ffbea4ecc93"
server_key_version: "encryptionkey@0"
I20260809 16:35:00.697786 9864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:00.722047 9864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:00.723819 9864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:00.723937 9864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:00.724123 9864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:00.724373 9864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:00.724403 9864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.724423 9864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:00.724434 9864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:00.729863 9864 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:40891
I20260809 16:35:00.729944 10067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:40891 every 8 connection(s)
I20260809 16:35:00.730207 9864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:35:00 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:00.736048 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 9864
I20260809 16:35:00.736136 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:00.736300 31807 external_mini_cluster.cc:1428] Setting key ac8f8b22bb5e3f93963aab27af15c32d
I20260809 16:35:00.738786 9362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.730866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:33097 (local address 127.31.15.254:40851)
0809 16:35:00.730938 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:00.730940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:00.731128 (+ 188us) server_negotiation.cc:407] Connection header received
0809 16:35:00.731679 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:00.731681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:00.731693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:00.731741 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:00.732674 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.733265 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.734266 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.734475 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.734531 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.736518 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:00.736520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:00.736521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:00.736529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:00.737754 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.738102 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.738103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.738103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.738150 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:00.738319 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:00.738320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:00.738320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:00.738429 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:00.738475 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:00.738666 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 16:35:00.738721 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:35:00.738967 10072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:00.730851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:33097)
0809 16:35:00.731014 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:00.731048 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:00.731577 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:00.731822 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:00.731825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:00.732390 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:00.732579 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.732583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.733342 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.733343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.734188 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:00.734190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:00.734543 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:00.734544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:00.734676 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:00.735092 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:00.735099 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:00.736434 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:00.737822 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.737828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.737830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.738039 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.738194 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:00.738195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:00.738195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:00.738280 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:00.738471 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:00.738479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:00.738582 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:35:00.738654 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:35:00.738765 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260809 16:35:00.739395 10068 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:00.739476 10068 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:00.739621 10068 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:00.739931 9293 ts_manager.cc:194] Registered new tserver with Master: 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:00.740260 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:33097
I20260809 16:35:00.747947 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.15.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.253@KRBTEST.COM" created.
I20260809 16:35:01.436899 9611 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
I20260809 16:35:01.581957 9855 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.253.keytab.
Entry for principal oryx/127.31.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.253.keytab.
W20260809 16:35:01.688192 31807 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.31.15.253: real 0.939s user 0.000s sys 0.004s
I20260809 16:35:01.688584 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:41239
--webserver_interface=127.31.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.253.keytab
--principal=oryx/127.31.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40851,127.31.15.253:41239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:35:01.741021 10068 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
W20260809 16:35:01.770557 10079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:01.770750 10079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:01.770781 10079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:01.771950 10079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:01.771998 10079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:01.772018 10079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:01.772034 10079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:01.772049 10079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:01.773526 10079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40851,127.31.15.253:41239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.253.keytab
--principal=oryx/127.31.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:41239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
W20260809 16:35:01.773635 10079 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.31.15.254:40851,127.31.15.253:41239'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 16:35:01.773788 10079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:01.774003 10079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:01.778520 10090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:01.778595 10087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:01.778528 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:01.778692 10079 server_base.cc:1034] running on GCE node
I20260809 16:35:01.779016 10079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:01.779291 10079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:01.780444 10079 hybrid_clock.cc:648] HybridClock initialized: now 1786293301780420 us; error 44 us; skew 500 ppm
Aug 09 16:35:01 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293301, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:01.782850 10079 init.cc:377] Logged in from keytab as oryx/127.31.15.253@KRBTEST.COM (short username oryx)
I20260809 16:35:01.783303 10079 webserver.cc:460] Webserver started at http://127.31.15.253:33927/ using document root <none> and password file <none>
I20260809 16:35:01.783571 10079 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:01.783617 10079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:01.783710 10079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:01.784682 10079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/instance:
uuid: "748c98777b13417c8a08a0828950f29c"
format_stamp: "Formatted at 2026-08-09 16:35:01 on dist-test-slave-21b9"
server_key: "a48dc0c62d789450ea43a4ccfa47b072"
server_key_iv: "193c1feba38c3809669fe2aa2ff2dd3a"
server_key_version: "encryptionkey@0"
I20260809 16:35:01.784970 10079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal/instance:
uuid: "748c98777b13417c8a08a0828950f29c"
format_stamp: "Formatted at 2026-08-09 16:35:01 on dist-test-slave-21b9"
server_key: "a48dc0c62d789450ea43a4ccfa47b072"
server_key_iv: "193c1feba38c3809669fe2aa2ff2dd3a"
server_key_version: "encryptionkey@0"
I20260809 16:35:01.786171 10079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:01.786877 10110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:01.787038 10079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:01.787122 10079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "748c98777b13417c8a08a0828950f29c"
format_stamp: "Formatted at 2026-08-09 16:35:01 on dist-test-slave-21b9"
server_key: "a48dc0c62d789450ea43a4ccfa47b072"
server_key_iv: "193c1feba38c3809669fe2aa2ff2dd3a"
server_key_version: "encryptionkey@0"
I20260809 16:35:01.787180 10079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:01.815238 10079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:01.817232 10079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:01.817355 10079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:01.821281 10079 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.253:41239
I20260809 16:35:01.821386 10189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.253:41239 every 8 connection(s)
I20260809 16:35:01.821667 10079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
I20260809 16:35:01.822654 10190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:01.823966 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 10079
I20260809 16:35:01.824263 10190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:35:01 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293301, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.253@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
Aug 09 16:35:01 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
I20260809 16:35:01.833909 10200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.825077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.31.15.254:40851)
0809 16:35:01.825201 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.825203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.825241 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:35:01.825449 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.825450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.825463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.825524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.826254 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.827065 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.828343 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.828648 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.828711 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.830965 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.830966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.830967 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.830975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.832466 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.832916 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.832917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.832917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.832982 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.833204 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.833204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.833205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.833390 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.833452 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.833675 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.833757 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260809 16:35:01.833922 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.824965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:42982)
0809 16:35:01.825130 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:01.825176 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:01.825349 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:01.825547 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:01.825549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:01.825921 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:01.826162 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.826169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.827182 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.827183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.828261 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.828264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.828726 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.828727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.828817 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.829284 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:01.829295 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:01.830877 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:01.832575 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.832581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.832583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.832849 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.833050 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.833051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.833051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.833153 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.833444 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:01.833447 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:01.833569 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 16:35:01.833645 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:35:01.833758 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260809 16:35:01.834649 10190 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:35:01.834985 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.824589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.31.15.253:41239)
0809 16:35:01.824761 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.824771 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.824814 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:35:01.825349 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.825357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.825378 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.825466 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.826259 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.827422 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.828388 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.828838 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.828885 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.831415 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.831423 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.831423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.831430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.832885 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.833961 (+ 1076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.833964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.833965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.834019 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.834313 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.834314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.834314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.834442 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.834587 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.834810 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.834878 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
Aug 09 16:35:01 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293301, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.253@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
I20260809 16:35:01.840729 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.834868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50234)
0809 16:35:01.834908 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:01.834917 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:01.834947 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:01.835140 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:01.835141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:01.835291 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:01.835342 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.835344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.835995 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.835997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.836474 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.836476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.836771 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.836771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.836818 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.837099 (+ 281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:01.837102 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:01.838136 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:01.840056 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.840057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.840057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.840197 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.840330 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.840331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.840331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.840372 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.840519 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:01.840519 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:01.840564 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:35:01.840607 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:35:01.840673 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:35:01.840828 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.834900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.31.15.253:41239)
0809 16:35:01.835041 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.835042 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.835048 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:01.835063 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.835064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.835073 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.835132 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.835406 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.835931 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.836538 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.836726 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.836766 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.838203 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.838204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.838205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.838210 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.839992 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.840247 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.840248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.840248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.840290 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.840404 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.840404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.840405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.840482 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.840522 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.840629 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.840717 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":125,"thread_start_us":104,"threads_started":1}
I20260809 16:35:01.841995 10190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Bootstrap starting.
I20260809 16:35:01.842599 10190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:01.842850 10190 log.cc:824] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:01.843582 10190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: No bootstrap required, opened a new log
I20260809 16:35:01.844909 10190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } }
I20260809 16:35:01.845028 10190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:01.845060 10190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748c98777b13417c8a08a0828950f29c, State: Initialized, Role: FOLLOWER
I20260809 16:35:01.845173 10190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } }
I20260809 16:35:01.845424 10190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:01.845389 10203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } }
I20260809 16:35:01.845475 10203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:35:01.846956 10220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:35:01.847010 10220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:35:01.848480 10079 master_runner.cc:428] Detected that this master 748c98777b13417c8a08a0828950f29c is joining an existing cluster
I20260809 16:35:01.848538 10079 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.15.253:41239
I20260809 16:35:01.856215 10200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.849530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42984 (local address 127.31.15.254:40851)
0809 16:35:01.849584 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.849586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.849651 (+ 65us) server_negotiation.cc:407] Connection header received
0809 16:35:01.849801 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.849802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.849812 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.849864 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.850331 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.851099 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.851988 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.852276 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.852332 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.853346 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.853347 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.853348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.853354 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.854815 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855245 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.855247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.855247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.855327 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855547 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.855548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.855548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.855674 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.855736 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.856017 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.856120 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:35:01.856415 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.849680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.31.15.253:41239)
0809 16:35:01.849710 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.849712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.849837 (+ 125us) server_negotiation.cc:407] Connection header received
0809 16:35:01.849911 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.849912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.849920 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.849943 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.850288 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.851026 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.852117 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.852406 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.852466 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.853763 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.853765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.853765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.853772 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.855425 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855734 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.855735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.855736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.855788 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855946 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.855947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.855947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.856045 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.856098 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.856221 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.856335 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":18}
I20260809 16:35:01.856534 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.849448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:42984)
0809 16:35:01.849583 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:01.849595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:01.849719 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:01.849868 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:01.849869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:01.850140 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:01.850262 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.850264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.851338 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.851339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.851925 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.851927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.852415 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.852416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.852479 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.852898 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:01.852902 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:01.853272 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:01.854895 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.854896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.854897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.855170 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.855401 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.855402 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.855477 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.855857 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:01.855858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:01.855899 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:35:01.856331 (+ 432us) client_negotiation.cc:240] Negotiation successful
0809 16:35:01.856428 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260809 16:35:01.856604 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.849651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50250)
0809 16:35:01.849748 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:01.849755 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:01.849856 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:01.849984 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:01.849985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:01.850167 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:01.850223 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.850224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.851111 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.851112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.852042 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.852053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.852474 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.852474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.852554 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.853037 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:01.853042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:01.853482 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:01.855484 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.855485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.855667 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.855844 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.855844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.855845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.855890 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.856098 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:01.856099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:01.856134 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:35:01.856197 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:35:01.856469 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
Aug 09 16:35:01 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.254@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
I20260809 16:35:01.865391 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.858590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50252)
0809 16:35:01.858713 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:01.858756 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:01.858819 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:01.858941 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:01.858943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:01.859181 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:01.859226 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.859227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.860097 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.860097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.860660 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:01.860661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.861086 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.861087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:01.861137 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.861538 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:01.861542 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:01.862969 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:01.864501 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.864509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.864510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.864696 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.864864 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:01.864865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:01.864865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:01.864911 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:01.865136 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:01.865140 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:01.865178 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 16:35:01.865235 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:35:01.865319 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260809 16:35:01.865428 10201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:01.858660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50252 (local address 127.31.15.253:41239)
0809 16:35:01.858697 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:01.858699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:01.858811 (+ 112us) server_negotiation.cc:407] Connection header received
0809 16:35:01.858870 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:01.858870 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:01.858879 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:01.858938 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:01.859317 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.860010 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.860726 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:01.861009 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:01.861089 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:01.863056 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:01.863057 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:01.863057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:01.863064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:01.864431 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.864756 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.864757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.864757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.864812 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:01.864956 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:01.864957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:01.864957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:01.865086 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:01.865147 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:01.865271 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:35:01.865337 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:35:01.866091 9293 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.15.253:41239: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "748c98777b13417c8a08a0828950f29c"
member_type: NON_VOTER
last_known_addr {
host: "127.31.15.253"
port: 41239
}
attrs {
promote: true
}
}
dest_uuid: "af165bb537bf4043aa288845e9f93043"
cas_config_opid_index: -1
I20260809 16:35:01.866387 9293 consensus_queue.cc:237] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } }
I20260809 16:35:01.867000 10235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 748c98777b13417c8a08a0828950f29c (127.31.15.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.867251 10156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:01.890511 10156 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 16:35:01.890501 10235 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.15.253:41239
I20260809 16:35:01.890740 10235 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.890815 10235 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:01.891232 10235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:01.893337 10156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 748c98777b13417c8a08a0828950f29c (127.31.15.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.894379 10203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader af165bb537bf4043aa288845e9f93043. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.894454 10203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:01.894492 10203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.894521 10203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:01.894639 10239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:01.894752 10239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:01.894915 10079 master.cc:537] Master@127.31.15.253:41239 shutting down...
I20260809 16:35:01.895597 10237 mvcc.cc:205] Tried to move back new op lower bound from 7316657357924073472 to 7316657357822070784. Current Snapshot: MvccSnapshot[applied={T|T < 7316657357832548352}]
I20260809 16:35:01.897899 10079 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Raft consensus shutting down.
I20260809 16:35:01.898020 10079 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Raft consensus is shut down!
I20260809 16:35:01.898066 10079 tablet_replica.cc:330] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: stopping tablet replica
WARNING: no policy specified for oryx/127.31.15.252@KRBTEST.COM; defaulting to no policy
I20260809 16:35:01.911258 10079 master.cc:552] Master@127.31.15.253:41239 shutdown complete.
I20260809 16:35:01.911341 10079 master_runner.cc:305] Clearing existing system tablet
I20260809 16:35:01.911480 10079 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:01.911533 10079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:01.912299 10079 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:01.912904 10250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:01.913089 10079 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:01.913148 10079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:01.913167 10079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "748c98777b13417c8a08a0828950f29c"
format_stamp: "Formatted at 2026-08-09 16:35:01 on dist-test-slave-21b9"
server_key: "a48dc0c62d789450ea43a4ccfa47b072"
server_key_iv: "193c1feba38c3809669fe2aa2ff2dd3a"
server_key_version: "encryptionkey@0"
I20260809 16:35:01.913769 10079 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:02.014750 10079 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:35:02.014874 10079 log.cc:1196] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 16:35:02.034415 10079 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Deleting consensus metadata
I20260809 16:35:02.034579 10079 master_runner.cc:315] Copying system tablet from 127.31.15.254:40851
I20260809 16:35:02.035179 10079 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: tablet copy: Beginning tablet copy session from remote peer at address 127.31.15.254:40851
I20260809 16:35:02.040762 10200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.035613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.31.15.254:40851)
0809 16:35:02.035676 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.035678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.035767 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:35:02.035835 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.035836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.035847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.035894 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.036226 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.036740 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.037442 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.037641 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.037702 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.038590 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.038592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.038592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.038599 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.039716 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.040018 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.040019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.040020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.040086 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.040247 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.040247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.040248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.040373 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.040429 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.040571 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.040661 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:35:02.040766 10264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.035512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:42992)
0809 16:35:02.035669 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.035682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.035775 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.035909 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.035910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.036096 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.036145 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.036147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.036819 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.036821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.037359 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.037362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.037704 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.037705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.037761 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.038194 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.038198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.038500 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.039790 (+ 1290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.039791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.039792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.039956 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.040142 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.040143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.040143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.040196 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.040437 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.040438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.040484 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.040539 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.040662 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260809 16:35:02.041225 9330 tablet_copy_service.cc:140] P af165bb537bf4043aa288845e9f93043: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 748c98777b13417c8a08a0828950f29c ({username='oryx', principal='oryx/127.31.15.253@KRBTEST.COM'} at 127.0.0.1:42992)
I20260809 16:35:02.041329 9330 tablet_copy_service.cc:161] P af165bb537bf4043aa288845e9f93043: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 748c98777b13417c8a08a0828950f29c at {username='oryx', principal='oryx/127.31.15.253@KRBTEST.COM'} at 127.0.0.1:42992: session id = 748c98777b13417c8a08a0828950f29c-00000000000000000000000000000000
I20260809 16:35:02.041975 9330 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:35:02.042585 10079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:35:02.043637 10079 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: tablet copy: Starting download of 0 data blocks...
I20260809 16:35:02.043781 10079 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: tablet copy: Starting download of 1 WAL segments...
I20260809 16:35:02.044670 10079 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:35:02.116765 9330 tablet_copy_service.cc:342] P af165bb537bf4043aa288845e9f93043: Request end of tablet copy session 748c98777b13417c8a08a0828950f29c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.15.253@KRBTEST.COM'} at 127.0.0.1:42992
I20260809 16:35:02.116864 9330 tablet_copy_service.cc:433] P af165bb537bf4043aa288845e9f93043: ending tablet copy session 748c98777b13417c8a08a0828950f29c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 748c98777b13417c8a08a0828950f29c
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.252@KRBTEST.COM" created.
I20260809 16:35:02.119625 10079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:02.120899 10269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.120908 10267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.121209 10266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:02.121410 10079 server_base.cc:1034] running on GCE node
I20260809 16:35:02.121495 10079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:02.121625 10079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:02.122737 10079 hybrid_clock.cc:648] HybridClock initialized: now 1786293302122734 us; error 27 us; skew 500 ppm
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:02.124091 10079 init.cc:377] Logged in from keytab as oryx/127.31.15.253@KRBTEST.COM (short username oryx)
I20260809 16:35:02.124342 10079 webserver.cc:460] Webserver started at http://127.31.15.253:44789/ using document root <none> and password file <none>
I20260809 16:35:02.124460 10079 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:02.124493 10079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:02.125062 10079 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:02.125557 10278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.125685 10079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.125730 10079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "748c98777b13417c8a08a0828950f29c"
format_stamp: "Formatted at 2026-08-09 16:35:01 on dist-test-slave-21b9"
server_key: "a48dc0c62d789450ea43a4ccfa47b072"
server_key_iv: "193c1feba38c3809669fe2aa2ff2dd3a"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.125788 10079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:02.136212 10079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:02.137360 10079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:02.140097 10079 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.253:41239
I20260809 16:35:02.141042 10354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.253:41239 every 8 connection(s)
I20260809 16:35:02.217427 10079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
I20260809 16:35:02.218385 10357 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:35:02.219089 10357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Bootstrap starting.
I20260809 16:35:02.254272 10357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:35:02.254478 10357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: Bootstrap complete.
I20260809 16:35:02.254693 10357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } }
I20260809 16:35:02.254769 10357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 748c98777b13417c8a08a0828950f29c, State: Initialized, Role: LEARNER
I20260809 16:35:02.254801 10357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } }
I20260809 16:35:02.254976 10357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:02.255010 10361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.255097 10361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:02.256376 10379 catalog_manager.cc:1259] Loaded cluster ID: e85e7a17a4e440bca1d5638ce5796862
I20260809 16:35:02.256428 10379 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: loading cluster ID for follower catalog manager: success
I20260809 16:35:02.257467 10379 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: acquiring CA information for follower catalog manager: success
I20260809 16:35:02.257733 10379 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:35:02.494999 10380 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.495152 10380 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:02.498169 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.495296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.31.15.253:41239)
0809 16:35:02.495471 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.495473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.495480 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:02.495504 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.495505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.495544 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.495613 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:02.495765 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.496433 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.497387 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.497799 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.497860 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.497939 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.498035 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.498077 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
I20260809 16:35:02.498155 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.495177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50264)
0809 16:35:02.495329 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.495341 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.495399 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.495641 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.495643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.495646 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:02.495701 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.495703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.496508 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.496509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.497296 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.497298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.497875 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.497876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.497937 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.497968 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.498027 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.498077 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":48,"threads_started":1}
I20260809 16:35:02.498526 10361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader af165bb537bf4043aa288845e9f93043. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.498634 10361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:02.498796 10383 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: attempting to promote NON_VOTER 748c98777b13417c8a08a0828950f29c to VOTER
I20260809 16:35:02.499025 10383 consensus_queue.cc:237] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } }
I20260809 16:35:02.499464 10320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 LEARNER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 16:35:02.499676 10380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 16:35:02.500452 10383 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 748c98777b13417c8a08a0828950f29c (127.31.15.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } }
I20260809 16:35:02.500633 10320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 748c98777b13417c8a08a0828950f29c (127.31.15.253) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } }
I20260809 16:35:02.501500 10383 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } }
I20260809 16:35:02.501564 10361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } }
I20260809 16:35:02.501578 10383 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:02.501560 10380 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } }
I20260809 16:35:02.501624 10361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:02.501643 10380 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.252.keytab.
Entry for principal oryx/127.31.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.252.keytab.
W20260809 16:35:02.515106 31807 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.31.15.252: real 0.623s user 0.000s sys 0.004s
I20260809 16:35:02.515501 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:37577
--webserver_interface=127.31.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.252.keytab
--principal=oryx/127.31.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:02.600442 10389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:02.600680 10389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:02.600701 10389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:02.601933 10389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:02.601979 10389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:02.601992 10389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:02.602003 10389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:02.602016 10389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:02.603570 10389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.252.keytab
--principal=oryx/127.31.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:37577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:02.603811 10389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:02.604039 10389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:02.608579 10397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.608597 10395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.608611 10402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:02.608876 10389 server_base.cc:1034] running on GCE node
I20260809 16:35:02.609113 10389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:02.609401 10389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:02.610549 10389 hybrid_clock.cc:648] HybridClock initialized: now 1786293302610533 us; error 31 us; skew 500 ppm
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:02.613025 10389 init.cc:377] Logged in from keytab as oryx/127.31.15.252@KRBTEST.COM (short username oryx)
I20260809 16:35:02.613469 10389 webserver.cc:460] Webserver started at http://127.31.15.252:36399/ using document root <none> and password file <none>
I20260809 16:35:02.613672 10389 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:02.613713 10389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:02.613801 10389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:02.614827 10389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/instance:
uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
format_stamp: "Formatted at 2026-08-09 16:35:02 on dist-test-slave-21b9"
server_key: "b972fc7d18c24f21bf9b83958f8f2fd8"
server_key_iv: "695b61033a0990f21902253c727a6369"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.615162 10389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal/instance:
uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
format_stamp: "Formatted at 2026-08-09 16:35:02 on dist-test-slave-21b9"
server_key: "b972fc7d18c24f21bf9b83958f8f2fd8"
server_key_iv: "695b61033a0990f21902253c727a6369"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.616411 10389 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:02.617139 10412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.617309 10389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:02.617430 10389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
format_stamp: "Formatted at 2026-08-09 16:35:02 on dist-test-slave-21b9"
server_key: "b972fc7d18c24f21bf9b83958f8f2fd8"
server_key_iv: "695b61033a0990f21902253c727a6369"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.617499 10389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:02.635849 10389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:02.637953 10389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:02.638099 10389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:02.641884 10389 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.252:37577
I20260809 16:35:02.641920 10496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.252:37577 every 8 connection(s)
I20260809 16:35:02.642243 10389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
I20260809 16:35:02.643190 10497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:02.644730 10497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.252@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:02.650879 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 10389
I20260809 16:35:02.653430 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.645513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.31.15.254:40851)
0809 16:35:02.645629 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.645631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.645682 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:35:02.646167 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.646168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.646184 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.646247 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.646849 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.647407 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.648402 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.648603 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.648646 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.650687 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.650689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.650689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.650697 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.652138 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.652494 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.652495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.652496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.652570 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.652813 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.652814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.652814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.652937 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.652993 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.653247 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.653332 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260809 16:35:02.653573 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.645375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43006)
0809 16:35:02.645552 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.645599 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.646072 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.646318 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.646321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.646572 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.646759 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.646763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.647498 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.647500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.648326 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.648328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.648664 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.648665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.648770 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.649222 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.649230 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.650598 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.652213 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.652219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.652221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.652431 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.652648 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.652649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.652649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.652758 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.652993 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.652999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.653133 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.653220 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.653367 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260809 16:35:02.654072 10497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.252@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
I20260809 16:35:02.659703 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.651407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.31.15.252:37577)
0809 16:35:02.651608 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.651613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.651874 (+ 261us) server_negotiation.cc:407] Connection header received
0809 16:35:02.651902 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.651905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.651918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.651983 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.652534 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.653292 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.654168 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.654635 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.654688 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.657035 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.657042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.657043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.657050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.658486 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.658817 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.658820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.658820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.658878 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.659042 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.659043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.659044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.659210 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.659366 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.659454 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.659516 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260809 16:35:02.660218 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.654316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50280)
0809 16:35:02.654360 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.654371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.654411 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.654527 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.654528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.654627 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.654668 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.654669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.655312 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.655313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.655861 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.655862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.656216 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.656217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.656274 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.656598 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.656603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.657704 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.659287 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.659288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.659289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.659486 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.659687 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.659687 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.659687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.659737 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.660008 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.660009 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.660043 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.660097 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.660163 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:35:02.660312 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.654373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.31.15.253:41239)
0809 16:35:02.654424 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.654425 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.654434 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:02.654459 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.654460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.654470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.654527 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.654736 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.655200 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.655925 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.656151 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.656206 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.657797 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.657799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.657799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.657808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.659190 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.659553 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.659554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.659554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.659617 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.659816 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.659817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.659818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.659943 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.660009 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.660134 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.660221 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:35:02.660686 10497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.252@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
I20260809 16:35:02.665592 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.660901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.0.0.1:53400)
0809 16:35:02.660983 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.660990 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.661028 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.661110 (+ 82us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.661111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.661209 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.661234 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.661235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.661761 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.661761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.662213 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.662214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.662495 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.662495 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.662527 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.662800 (+ 273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.662804 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.663823 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.664918 (+ 1095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.664919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.664919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.665065 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.665196 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.665196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.665196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.665237 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.665383 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.665383 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.665413 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.665454 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.665516 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 16:35:02.665643 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.660946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.31.15.252:37577)
0809 16:35:02.660991 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.660992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.661018 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:35:02.661068 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.661069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.661076 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.661107 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.661269 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.661699 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.662273 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.662449 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.662492 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.663899 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.663900 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.663900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.663906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.664859 (+ 953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.665115 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.665116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.665116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.665155 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.665269 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.665269 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.665270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.665346 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.665386 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.665472 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.665557 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":29}
I20260809 16:35:02.666934 10497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Bootstrap starting.
I20260809 16:35:02.667734 10497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:02.667950 10497 log.cc:824] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:02.668596 10497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: No bootstrap required, opened a new log
I20260809 16:35:02.669976 10497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } }
I20260809 16:35:02.670095 10497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:02.670117 10497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a7ee897e0ca48ea87575b1bb522f45c, State: Initialized, Role: FOLLOWER
I20260809 16:35:02.670215 10497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } }
I20260809 16:35:02.670444 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } } }
I20260809 16:35:02.670490 10497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:02.670536 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:35:02.672073 10523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:35:02.672132 10523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:35:02.674316 10389 master_runner.cc:428] Detected that this master 7a7ee897e0ca48ea87575b1bb522f45c is joining an existing cluster
I20260809 16:35:02.674355 10389 master_runner.cc:432] Initiating AddMaster RPC to add 127.31.15.252:37577
I20260809 16:35:02.703208 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.675885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50282 (local address 127.31.15.253:41239)
0809 16:35:02.675999 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.676001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.676211 (+ 210us) server_negotiation.cc:407] Connection header received
0809 16:35:02.676295 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.676296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.676305 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.676330 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.676716 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.677446 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678316 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.678570 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678627 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.700134 (+ 21507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.700136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.700136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.700145 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.701764 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702339 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.702395 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702618 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.702757 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.702797 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.703000 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.703102 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 16:35:02.703375 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.675345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53412 (local address 127.31.15.252:37577)
0809 16:35:02.675384 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.675386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.675527 (+ 141us) server_negotiation.cc:407] Connection header received
0809 16:35:02.675636 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.675637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.675646 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.675689 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.676068 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.676731 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.677701 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.677981 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678015 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.700017 (+ 22002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.700019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.700019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.700028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.702359 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702711 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.702752 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702907 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.703011 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.703047 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.703195 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.703257 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:35:02.703375 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.675583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.31.15.254:40851)
0809 16:35:02.675624 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.675626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.675821 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:35:02.675914 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.675915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.675928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.675957 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.676374 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.677145 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678204 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.678826 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678873 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.700211 (+ 21338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.700212 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.700213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.700222 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.701960 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702535 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.702592 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702783 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.702783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.702784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.702934 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.702976 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.703158 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.703254 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:35:02.703745 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.675649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43016)
0809 16:35:02.675767 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.675776 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.675844 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.676016 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.676017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.676221 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.676276 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.676278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.677242 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.677243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.678102 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.678104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.678954 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.679045 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.679399 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.679404 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.700069 (+ 20665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.702196 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.702197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702451 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.702661 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702661 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.702661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702715 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.703019 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.703019 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.703055 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.703530 (+ 475us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.703648 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260809 16:35:02.703859 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.675304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.0.0.1:53412)
0809 16:35:02.675452 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.675463 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.675526 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.675697 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.675698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.675883 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.675985 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.675987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.676804 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.676804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.677629 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.677630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.678077 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.678133 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.678567 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.678572 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.699833 (+ 21261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.702451 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.702452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702644 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.702810 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.702811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702853 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.703081 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.703082 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.703111 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.703689 (+ 578us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.703763 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 16:35:02.704208 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.676029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50282)
0809 16:35:02.676160 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.676169 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.676234 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.676383 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.676383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.676580 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.676637 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.676638 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.677533 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.677534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.678257 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.678259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.678679 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.678680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.678737 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.679087 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.679091 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.699965 (+ 20874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.701860 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.701861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.701862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702128 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.702464 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.702464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.702465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.702550 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.702839 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.702840 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.702895 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.703896 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.704106 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293300, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.254@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
I20260809 16:35:02.711158 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 9402
I20260809 16:35:02.712074 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.705803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.0.0.1:53424)
0809 16:35:02.705898 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.705909 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.705957 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.706074 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.706075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.706209 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.706243 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.706245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.706806 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.706807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.707353 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.707355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.707669 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.707670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.707731 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.708088 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.708092 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.709499 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.710974 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.710975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.710976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.711167 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.711403 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.711404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.711404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.711471 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.711748 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.711749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.711797 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.711855 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.711960 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 16:35:02.712415 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.705853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53424 (local address 127.31.15.252:37577)
0809 16:35:02.705888 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.705890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.705942 (+ 52us) server_negotiation.cc:407] Connection header received
0809 16:35:02.706012 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.706012 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.706022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.706065 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.706304 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.706742 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.707426 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.707612 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.707665 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.709598 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.709599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.709599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.709605 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.710909 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.711241 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.711242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.711242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.711324 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.711535 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.711535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.711536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.711669 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.711724 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.712256 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.712329 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:35:02.741983 9293 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.31.15.252:37577: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
member_type: NON_VOTER
last_known_addr {
host: "127.31.15.252"
port: 37577
}
attrs {
promote: true
}
}
dest_uuid: "af165bb537bf4043aa288845e9f93043"
cas_config_opid_index: 6
I20260809 16:35:02.742197 9293 consensus_queue.cc:237] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } }
I20260809 16:35:02.742758 10320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 16:35:02.742828 10456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:02.742977 10380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 16:35:02.743688 10380 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.743854 10456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 16:35:02.744001 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:45009
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:40851
--webserver_port=37275
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:35:02.744102 10382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:02.744673 10380 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.31.15.252:37577
I20260809 16:35:02.744743 10380 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.744798 10380 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:02.744841 10380 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.744875 10380 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:02.745118 10320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.745519 10456 raft_consensus.cc:858] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } }
W20260809 16:35:02.745587 10456 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } }
I20260809 16:35:02.746098 10361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.746181 10361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:02.746116 10456 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 748c98777b13417c8a08a0828950f29c (127.31.15.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.746471 10389 master.cc:537] Master@127.31.15.252:37577 shutting down...
I20260809 16:35:02.746840 10542 mvcc.cc:205] Tried to move back new op lower bound from 7316657357924073472 to 7316657357822070784. Current Snapshot: MvccSnapshot[applied={T|T < 7316657357832548352}]
I20260809 16:35:02.747005 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: New leader af165bb537bf4043aa288845e9f93043. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.747097 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 16:35:02.747131 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.747162 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 16:35:02.747184 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: NON_VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: true } } }
I20260809 16:35:02.747215 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 16:35:02.748855 10383 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.748946 10383 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
W20260809 16:35:02.749150 9270 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 16:35:02.749617 9270 consensus_peers.cc:487] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 -> Peer 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252:37577): Couldn't send request to peer 7a7ee897e0ca48ea87575b1bb522f45c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 16:35:02.749820 10389 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 16:35:02.749922 10389 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 16:35:02.749963 10389 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: stopping tablet replica
I20260809 16:35:02.763677 10389 master.cc:552] Master@127.31.15.252:37577 shutdown complete.
I20260809 16:35:02.763741 10389 master_runner.cc:305] Clearing existing system tablet
I20260809 16:35:02.763875 10389 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:02.763926 10389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:02.764729 10389 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:02.765273 10562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.765378 10389 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:02.765426 10389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:02.765448 10389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
format_stamp: "Formatted at 2026-08-09 16:35:02 on dist-test-slave-21b9"
server_key: "b972fc7d18c24f21bf9b83958f8f2fd8"
server_key_iv: "695b61033a0990f21902253c727a6369"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.766235 10389 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:02.767678 10389 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 16:35:02.767751 10389 log.cc:1196] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 16:35:02.767987 10389 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Deleting consensus metadata
I20260809 16:35:02.768106 10389 master_runner.cc:315] Copying system tablet from 127.31.15.254:40851
I20260809 16:35:02.768723 10389 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: tablet copy: Beginning tablet copy session from remote peer at address 127.31.15.254:40851
I20260809 16:35:02.775966 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.769091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43018)
0809 16:35:02.769215 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.769227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.769305 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.769471 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.769472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.769679 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.769741 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.769743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.770661 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.770662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.771480 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.771483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.771993 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.771994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.772066 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.772566 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.772572 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.772955 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.774837 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.774838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.774839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.775087 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.775321 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.775322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.775322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.775387 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.775648 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.775649 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.775696 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.775763 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.775868 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":79,"threads_started":1}
I20260809 16:35:02.775987 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.769091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.31.15.254:40851)
0809 16:35:02.769157 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.769159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.769302 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:35:02.769384 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.769386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.769397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.769448 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.769813 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.770568 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.771588 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.771915 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.771991 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.773056 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.773058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.773058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.773066 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.774761 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.775147 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.775148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.775148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.775214 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.775450 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.775451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.775451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.775586 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.775654 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.775799 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.775876 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:35:02.776584 9330 tablet_copy_service.cc:140] P af165bb537bf4043aa288845e9f93043: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7a7ee897e0ca48ea87575b1bb522f45c ({username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:43018)
I20260809 16:35:02.776650 9330 tablet_copy_service.cc:161] P af165bb537bf4043aa288845e9f93043: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7a7ee897e0ca48ea87575b1bb522f45c at {username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:43018: session id = 7a7ee897e0ca48ea87575b1bb522f45c-00000000000000000000000000000000
I20260809 16:35:02.777200 9330 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 16:35:02.777679 10389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 16:35:02.778910 10389 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: tablet copy: Starting download of 0 data blocks...
I20260809 16:35:02.779052 10389 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: tablet copy: Starting download of 1 WAL segments...
I20260809 16:35:02.779831 10389 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 16:35:02.780702 9330 tablet_copy_service.cc:342] P af165bb537bf4043aa288845e9f93043: Request end of tablet copy session 7a7ee897e0ca48ea87575b1bb522f45c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:43018
I20260809 16:35:02.780752 9330 tablet_copy_service.cc:433] P af165bb537bf4043aa288845e9f93043: ending tablet copy session 7a7ee897e0ca48ea87575b1bb522f45c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7a7ee897e0ca48ea87575b1bb522f45c
I20260809 16:35:02.783028 10389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:02.784199 10581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.784247 10588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.784399 10582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:02.784524 10389 server_base.cc:1034] running on GCE node
I20260809 16:35:02.784600 10389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:02.784718 10389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:02.785845 10389 hybrid_clock.cc:648] HybridClock initialized: now 1786293302785836 us; error 23 us; skew 500 ppm
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:02.787413 10389 init.cc:377] Logged in from keytab as oryx/127.31.15.252@KRBTEST.COM (short username oryx)
I20260809 16:35:02.787680 10389 webserver.cc:460] Webserver started at http://127.31.15.252:45611/ using document root <none> and password file <none>
I20260809 16:35:02.787786 10389 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:02.787813 10389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:02.788493 10389 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:02.789038 10597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.789213 10389 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.789263 10389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
format_stamp: "Formatted at 2026-08-09 16:35:02 on dist-test-slave-21b9"
server_key: "b972fc7d18c24f21bf9b83958f8f2fd8"
server_key_iv: "695b61033a0990f21902253c727a6369"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.789321 10389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:02.810633 10389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:02.812527 10389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:02.816238 10389 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.252:37577
I20260809 16:35:02.816533 10676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.252:37577 every 8 connection(s)
I20260809 16:35:02.816679 10389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
I20260809 16:35:02.817543 10678 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:35:02.818050 10678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Bootstrap starting.
I20260809 16:35:02.820427 10678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:35:02.820714 10678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Bootstrap complete.
I20260809 16:35:02.820958 10678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } }
I20260809 16:35:02.821074 10678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7a7ee897e0ca48ea87575b1bb522f45c, State: Initialized, Role: LEARNER
I20260809 16:35:02.821113 10678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } }
I20260809 16:35:02.821280 10678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:02.822590 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:02.822688 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:02.823431 10704 catalog_manager.cc:1259] Loaded cluster ID: e85e7a17a4e440bca1d5638ce5796862
I20260809 16:35:02.823513 10704 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: loading cluster ID for follower catalog manager: success
I20260809 16:35:02.824502 10704 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: acquiring CA information for follower catalog manager: success
I20260809 16:35:02.824762 10704 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 16:35:02.838784 10543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:02.838970 10543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:02.838991 10543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:02.840163 10543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:02.840204 10543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:02.840231 10543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:02.841663 10543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:45009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=37275
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:02.841878 10543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:02.842095 10543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:02.846650 10713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.846685 10714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.846649 10718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:02.846946 10543 server_base.cc:1034] running on GCE node
I20260809 16:35:02.847167 10543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:02.847417 10543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:02.848560 10543 hybrid_clock.cc:648] HybridClock initialized: now 1786293302848547 us; error 28 us; skew 500 ppm
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:02.851236 10543 init.cc:377] Logged in from keytab as oryx/127.31.15.193@KRBTEST.COM (short username oryx)
I20260809 16:35:02.851667 10543 webserver.cc:460] Webserver started at http://127.31.15.193:37275/ using document root <none> and password file <none>
I20260809 16:35:02.851871 10543 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:02.851933 10543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:02.853276 10543 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:02.853904 10727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.854079 10543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:35:02.854161 10543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "fd4eba99d45d4d799f5b5381e7758c81"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "60f1e6554de84c2470abdab42042d468"
server_key_iv: "fad76f7fa0f425c48a6bdc0c2704af21"
server_key_version: "encryptionkey@0"
I20260809 16:35:02.854468 10543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:02.867532 10543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:02.869313 10543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:02.869450 10543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:02.869674 10543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:02.869961 10543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:02.869995 10543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.870024 10543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:02.870059 10543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:02.876030 10543 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45009
I20260809 16:35:02.876104 10920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45009 every 8 connection(s)
I20260809 16:35:02.876423 10543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
I20260809 16:35:02.883210 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 10543
I20260809 16:35:02.883412 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 9621
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:02.895344 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:33985 (local address 127.31.15.254:40851)
0809 16:35:02.877657 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.877659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.878135 (+ 476us) server_negotiation.cc:407] Connection header received
0809 16:35:02.878224 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.878225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.878240 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.878303 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.878931 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.879549 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.880881 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881091 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881145 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.889283 (+ 8138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.889286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.889286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.889296 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.891241 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.894097 (+ 2856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.894099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.894099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.894184 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.894568 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.894569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.894569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.894727 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.894775 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.895146 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.895230 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:35:02.895859 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:33985)
0809 16:35:02.878080 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.878089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.878158 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.878324 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.878324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.878603 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.878828 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.878834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.879630 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.879631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.880773 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.880775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881196 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.881285 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.881849 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.881859 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.888577 (+ 6718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.891376 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.891385 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.891388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.891777 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.894288 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.894289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.894289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.894473 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.894823 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.894831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.895004 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.895476 (+ 472us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.895647 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":58,"thread_start_us":54,"threads_started":1}
I20260809 16:35:02.896268 10923 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:02.896347 10923 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:02.896543 10923 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:02.896919 9293 ts_manager.cc:194] Re-registered known tserver with Master: fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009)
I20260809 16:35:02.897322 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:33985
I20260809 16:35:02.898849 10932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.31.15.193:40585)
0809 16:35:02.877619 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.877628 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.877928 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.878175 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.878179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.878629 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.878828 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.878832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.880142 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.880143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.880771 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.880773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881462 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.881530 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.882030 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.882035 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.888202 (+ 6167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.893679 (+ 5477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.893681 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.893682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.893968 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.898219 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.898220 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.898220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.898276 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.898565 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.898566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.898601 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.898653 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.898749 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":5,"spinlock_wait_cycles":7424,"thread_start_us":100,"threads_started":1}
I20260809 16:35:02.899210 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:40585 (local address 127.31.15.252:37577)
0809 16:35:02.877537 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.877539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.877696 (+ 157us) server_negotiation.cc:407] Connection header received
0809 16:35:02.878011 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.878012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.878027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.878098 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.879351 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.880064 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881188 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881390 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881452 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.888696 (+ 7244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.888698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.888699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.888708 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.893558 (+ 4850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.898069 (+ 4511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.898071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.898071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.898143 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.898363 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.898364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.898364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.898487 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.898536 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.899038 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.899128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260809 16:35:02.904146 10922 heartbeater.cc:347] Connected to a master server at 127.31.15.252:37577
I20260809 16:35:02.904217 10922 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:02.904307 10922 heartbeater.cc:511] Master 127.31.15.252:37577 requested a full tablet report, sending...
I20260809 16:35:02.904789 10626 ts_manager.cc:194] Registered new tserver with Master: fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009)
I20260809 16:35:02.907727 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:37055 (local address 127.31.15.253:41239)
0809 16:35:02.877173 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:02.877175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:02.877442 (+ 267us) server_negotiation.cc:407] Connection header received
0809 16:35:02.878016 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:02.878017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:02.878031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:02.878093 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:02.878940 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.879481 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.880863 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881073 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881132 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.904906 (+ 23774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:02.904908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:02.904909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:02.904919 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:02.906641 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.906946 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.906947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.906948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.907005 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:02.907244 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:02.907245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:02.907245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:02.907390 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:02.907441 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:02.907598 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:35:02.907664 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:35:02.907817 10927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:02.877143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.31.15.193:37055)
0809 16:35:02.877324 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:02.877360 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:02.877919 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:02.878174 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:02.878179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:02.878601 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:02.878828 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.878834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.879584 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.879585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.880772 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:02.880775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:02.881152 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:02.881153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:02.881285 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:02.881902 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:02.881905 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:02.904807 (+ 22902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:02.906720 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.906722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.906722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.906883 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.907089 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:02.907090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:02.907091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:02.907175 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:02.907449 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:02.907450 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:02.907501 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 16:35:02.907583 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:35:02.907706 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":39,"thread_start_us":98,"threads_started":1}
I20260809 16:35:02.908228 10924 heartbeater.cc:347] Connected to a master server at 127.31.15.253:41239
I20260809 16:35:02.908283 10924 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:02.908332 10924 heartbeater.cc:511] Master 127.31.15.253:41239 requested a full tablet report, sending...
I20260809 16:35:02.908695 10302 ts_manager.cc:194] Registered new tserver with Master: fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009)
I20260809 16:35:02.908969 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:36171
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:40851
--webserver_port=45543
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:02.990587 10936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:02.990787 10936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:02.990818 10936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:02.991941 10936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:02.991979 10936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:02.992014 10936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:02.993464 10936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:36171
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=45543
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:02.993712 10936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:02.993920 10936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:02.998811 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.998771 10951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:02.998771 10944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:02.999125 10936 server_base.cc:1034] running on GCE node
I20260809 16:35:02.999361 10936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:02.999600 10936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:03.000782 10936 hybrid_clock.cc:648] HybridClock initialized: now 1786293303000742 us; error 30 us; skew 500 ppm
Aug 09 16:35:02 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:03.003480 10936 init.cc:377] Logged in from keytab as oryx/127.31.15.194@KRBTEST.COM (short username oryx)
I20260809 16:35:03.003980 10936 webserver.cc:460] Webserver started at http://127.31.15.194:45543/ using document root <none> and password file <none>
I20260809 16:35:03.004197 10936 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:03.004271 10936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:03.005673 10936 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:03.006271 10960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.006464 10936 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:03.006567 10936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "2af9bdf77e0d472fa940ef53366c1775"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "df7d2900f4d8ef691312ff157df7ce89"
server_key_iv: "e5a9fbdea896516c78dab35fab466c38"
server_key_version: "encryptionkey@0"
I20260809 16:35:03.006933 10936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:03.022408 10936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:03.024526 10936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:03.024667 10936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:03.024853 10936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:03.025180 10936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:03.025218 10936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.025246 10936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:03.025264 10936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.031127 10936 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:36171
I20260809 16:35:03.031174 11148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:36171 every 8 connection(s)
I20260809 16:35:03.031540 10936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:35:03.034107 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 10936
I20260809 16:35:03.034279 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 9864
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
I20260809 16:35:03.042460 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:34925 (local address 127.31.15.253:41239)
0809 16:35:03.032840 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.032842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.032853 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:35:03.033940 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.033941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.033955 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.034009 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.034753 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.035395 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.036653 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.036907 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.036949 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.039327 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.039329 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.039329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.039338 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.040768 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.041224 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.041225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.041226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.041278 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.041518 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.041518 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.041519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.041644 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.041687 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.042278 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.042367 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:35:03.042981 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.31.15.194:34925)
0809 16:35:03.032719 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.032730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.033532 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.034060 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.034061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.034370 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.034642 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.034648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.035500 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.035502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.036555 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.036558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.037036 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.037037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.037160 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.037694 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.037703 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.039194 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.040858 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.040865 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.040868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.041146 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.041340 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.041341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.041341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.041454 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.041728 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.041733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.041892 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.042586 (+ 694us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.042755 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:03.047552 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40027 (local address 127.31.15.254:40851)
0809 16:35:03.032326 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.032328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.032574 (+ 246us) server_negotiation.cc:407] Connection header received
0809 16:35:03.033641 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.033643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.033659 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.033724 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.038981 (+ 5257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.039955 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.041196 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.041544 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.041592 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.044080 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.044082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.044082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.044092 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.046005 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.046501 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.046502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.046503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.046573 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.046820 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.046821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.046822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.046989 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.047042 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.047342 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.047451 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:35:03.047976 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:40027)
0809 16:35:03.032443 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.032486 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.033530 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.033799 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.033803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.038795 (+ 4992us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.038870 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.038872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.040078 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.040079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.041075 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.041079 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.041683 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.041684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.041765 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.042169 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.042175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.043936 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.046131 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.046132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.046133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.046401 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.046661 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.046662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.046662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.046730 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.047110 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.047112 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.047168 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.047700 (+ 532us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.047851 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":136,"threads_started":1}
I20260809 16:35:03.052464 11153 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:03.052603 11153 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.052819 11153 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:03.052997 11154 heartbeater.cc:347] Connected to a master server at 127.31.15.253:41239
I20260809 16:35:03.053028 11154 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.053076 11154 heartbeater.cc:511] Master 127.31.15.253:41239 requested a full tablet report, sending...
I20260809 16:35:03.053246 9293 ts_manager.cc:194] Re-registered known tserver with Master: 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171)
I20260809 16:35:03.053375 10302 ts_manager.cc:194] Registered new tserver with Master: 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171)
I20260809 16:35:03.053761 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:40027
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293302, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
I20260809 16:35:03.056350 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:40891
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:40851
--webserver_port=34811
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:35:03.058835 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:57361 (local address 127.31.15.252:37577)
0809 16:35:03.032309 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.032312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.032574 (+ 262us) server_negotiation.cc:407] Connection header received
0809 16:35:03.033797 (+ 1223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.033798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.033811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.033874 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.050468 (+ 16594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.051484 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.053310 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.053597 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.053646 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.056105 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.056107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.056107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.056115 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.057672 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.058090 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.058091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.058091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.058153 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.058389 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.058389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.058389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.058497 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.058549 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.058695 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.058768 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:35:03.059116 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.032260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.31.15.194:57361)
0809 16:35:03.032443 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.032485 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.033516 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.033887 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.033888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.038569 (+ 4681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.038646 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.038649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.052541 (+ 13892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.052542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.053205 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.053208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.053737 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.053738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.053802 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.054482 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.054486 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.055989 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.057780 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.057782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.057782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.057998 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.058213 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.058214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.058214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.058283 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.058554 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.058555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.058603 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.058679 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.058804 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":107,"threads_started":1}
I20260809 16:35:03.059506 11149 heartbeater.cc:347] Connected to a master server at 127.31.15.252:37577
I20260809 16:35:03.059572 11149 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.059652 11149 heartbeater.cc:511] Master 127.31.15.252:37577 requested a full tablet report, sending...
I20260809 16:35:03.059999 10626 ts_manager.cc:194] Registered new tserver with Master: 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171)
W20260809 16:35:03.139971 11159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:03.140239 11159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:03.140266 11159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:03.141423 11159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:03.141469 11159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:03.141496 11159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:03.142978 11159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:40891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=34811
--tserver_master_addrs=127.31.15.254:40851,127.31.15.253:41239,127.31.15.252:37577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:03.143198 11159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:03.143457 11159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:03.148242 11168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:03.148233 11171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:03.148233 11169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:03.148507 11159 server_base.cc:1034] running on GCE node
I20260809 16:35:03.148733 11159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:03.148972 11159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:03.150130 11159 hybrid_clock.cc:648] HybridClock initialized: now 1786293303150108 us; error 35 us; skew 500 ppm
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293303, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:03.152755 11159 init.cc:377] Logged in from keytab as oryx/127.31.15.195@KRBTEST.COM (short username oryx)
I20260809 16:35:03.153174 11159 webserver.cc:460] Webserver started at http://127.31.15.195:34811/ using document root <none> and password file <none>
I20260809 16:35:03.153409 11159 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:03.153475 11159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:03.154868 11159 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:03.155516 11180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.155696 11159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:03.155788 11159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
format_stamp: "Formatted at 2026-08-09 16:35:00 on dist-test-slave-21b9"
server_key: "86a5a108917415b9bc10810d853fe907"
server_key_iv: "c6ec1107d30d82a1118e4ffbea4ecc93"
server_key_version: "encryptionkey@0"
I20260809 16:35:03.156134 11159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:03.169489 11159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:03.171360 11159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:03.171489 11159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:03.171691 11159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:03.171978 11159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:03.172020 11159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.172049 11159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:03.172075 11159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:03.177591 11159 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:40891
I20260809 16:35:03.177651 11364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:40891 every 8 connection(s)
I20260809 16:35:03.177953 11159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:03.182287 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 11159
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293303, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.253@KRBTEST.COM
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293303, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.252@KRBTEST.COM
Aug 09 16:35:03 dist-test-slave-21b9 krb5kdc[9224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293303, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:03.187885 10382 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:03.188017 10382 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:03.191250 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.178666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:49027 (local address 127.31.15.253:41239)
0809 16:35:03.178764 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.178766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.178953 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:35:03.180035 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.180036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.180072 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.180124 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.180828 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.181609 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.182963 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183228 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183291 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.187159 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.187161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.187161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.187169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.188725 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.189203 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.189204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.189204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.189262 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.189529 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.189530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.189531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.189654 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.189712 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.191071 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.191156 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 16:35:03.191706 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.178662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.31.15.195:49027)
0809 16:35:03.178835 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.178876 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.179914 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.180202 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.180206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.180533 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.180734 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.180738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.181700 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.181700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.182856 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.182860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183384 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.183537 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.184143 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.184152 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.187060 (+ 2908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.188819 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.188826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.188829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.189101 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.189333 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.189334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.189334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.189478 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.190755 (+ 1277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.190763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.190929 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.191043 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.191206 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":46,"thread_start_us":84,"threads_started":1}
I20260809 16:35:03.192327 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.183565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.31.15.252:37577)
0809 16:35:03.184438 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.184440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.184447 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:03.184467 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.184468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.184479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.184567 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.185346 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.186076 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.187145 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.187472 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.187526 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.189075 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.189076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.189076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.189083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.190978 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.191383 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.191384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.191385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.191444 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.191621 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.191622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.191622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.191740 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.191838 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.192159 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.192238 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":63,"threads_started":1}
I20260809 16:35:03.192355 11368 heartbeater.cc:347] Connected to a master server at 127.31.15.253:41239
I20260809 16:35:03.192451 11368 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.192654 11368 heartbeater.cc:511] Master 127.31.15.253:41239 requested a full tablet report, sending...
I20260809 16:35:03.193054 10302 ts_manager.cc:194] Registered new tserver with Master: 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:03.193187 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.178663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.31.15.195:56209)
0809 16:35:03.178833 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.178876 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.179946 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.180211 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.180212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.180534 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.180742 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.180743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.182554 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.182555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.183306 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.183308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183799 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.183855 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.184254 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.184259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.189773 (+ 5514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.191740 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.191740 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.191741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.191938 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.192224 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.192225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.192277 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.192883 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.192884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.192926 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.192983 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.193083 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260809 16:35:03.193269 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.179050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:50551 (local address 127.31.15.254:40851)
0809 16:35:03.179108 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.179110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.179119 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:03.180066 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.180067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.180080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.180113 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.180864 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.181617 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.182946 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183260 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183329 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.187023 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.187024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.187024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.187043 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.189236 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192434 (+ 3198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.192435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.192435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.192499 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192702 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.192703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.192704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.192825 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.192883 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.193107 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.193187 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:35:03.193480 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.178657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56209 (local address 127.31.15.252:37577)
0809 16:35:03.178725 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.178728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.178953 (+ 225us) server_negotiation.cc:407] Connection header received
0809 16:35:03.180076 (+ 1123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.180077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.180088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.180126 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.181707 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.182457 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183369 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183672 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183740 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.189855 (+ 6115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.189856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.189857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.189863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.191662 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192020 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.192021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.192021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.192078 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192347 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.192348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.192348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.192466 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.192520 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.193320 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.193394 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:35:03.193920 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.178947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:50551)
0809 16:35:03.179070 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.179079 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.179941 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.180210 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.180211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.180572 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:03.180735 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.180741 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.181720 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.181720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.182851 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.182854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.183408 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.183409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.183537 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.184566 (+ 1029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:03.184570 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:03.186922 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:03.192186 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.192187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.192375 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.192576 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:03.192577 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:03.192578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:03.192637 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:03.192911 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:03.192911 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:03.192964 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.193022 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.193823 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":729,"thread_start_us":55,"threads_started":1}
I20260809 16:35:03.193972 11365 heartbeater.cc:347] Connected to a master server at 127.31.15.252:37577
I20260809 16:35:03.194005 11365 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.194062 11365 heartbeater.cc:511] Master 127.31.15.252:37577 requested a full tablet report, sending...
I20260809 16:35:03.194284 11366 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40851
I20260809 16:35:03.194296 10626 ts_manager.cc:194] Registered new tserver with Master: 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:03.194334 11366 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:03.194401 11366 heartbeater.cc:511] Master 127.31.15.254:40851 requested a full tablet report, sending...
I20260809 16:35:03.194725 9293 ts_manager.cc:194] Re-registered known tserver with Master: 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:03.195160 9293 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:50551
I20260809 16:35:03.195315 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.188430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.31.15.252:37577)
0809 16:35:03.189591 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.189593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.189601 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:03.189620 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.189621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.189652 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.189899 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.189929 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.190689 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.194429 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.194980 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.195048 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.195122 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.195187 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.195224 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":61,"threads_started":1}
I20260809 16:35:03.195541 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.188194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.0.0.1:53436)
0809 16:35:03.188312 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.188321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.188359 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.189759 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.189760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.189763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.189825 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.189827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.190785 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.190786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.194337 (+ 3551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.194339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.195060 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.195061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.195118 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.195122 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.195181 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.195221 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":2433}
I20260809 16:35:03.196069 10642 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260809 16:35:03.196213 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: New leader af165bb537bf4043aa288845e9f93043. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: NON_VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: true } } }
I20260809 16:35:03.196291 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: LEARNER
I20260809 16:35:03.196557 10383 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: attempting to promote NON_VOTER 7a7ee897e0ca48ea87575b1bb522f45c to VOTER
I20260809 16:35:03.196712 10383 consensus_queue.cc:237] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:03.197116 10320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 16:35:03.197312 10382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 16:35:03.197397 10642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 LEARNER]: Refusing update from remote peer af165bb537bf4043aa288845e9f93043: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 16:35:03.197623 10382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 16:35:03.198014 10382 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.198191 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.184213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.31.15.253:41239)
0809 16:35:03.185364 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.185366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.185505 (+ 139us) server_negotiation.cc:407] Connection header received
0809 16:35:03.185537 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.185538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.185550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.185617 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.186184 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.186912 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.188829 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.189170 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.189231 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.190497 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.190499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.190499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.190506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.193204 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.196450 (+ 3246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.196451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.196451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.196513 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.196703 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.196703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.196704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.196864 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.197963 (+ 1099us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.198004 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.198091 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1130,"thread_start_us":58,"threads_started":1}
I20260809 16:35:03.198835 10382 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.198912 10382 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:03.198959 10382 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.199004 10382 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:03.199198 10320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.199487 10383 sys_catalog.cc:455] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.199568 10383 sys_catalog.cc:458] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:03.199702 10642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.200112 10361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.200193 10361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:03.200359 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.184303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.31.15.254:40851)
0809 16:35:03.185239 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.185240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.185248 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:03.185270 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.185271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.185282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.185340 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.186009 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.193596 (+ 7587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.194908 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.195140 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.195191 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.197272 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.197273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.197274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.197280 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.198910 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.199262 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.199263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.199264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.199389 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.199596 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.199597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.199597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.199716 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.199806 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.200022 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.200277 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":908,"thread_start_us":72,"threads_started":1}
I20260809 16:35:03.200664 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af165bb537bf4043aa288845e9f93043" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:03.200745 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:03.201582 9293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:03.203109 9293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:03.209821 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40891 (local address 127.0.0.1:43522)
0809 16:35:03.206546 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.206556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.206595 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.206818 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.206818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.206820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.206856 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.206857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208107 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.208108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.208902 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.208906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.209627 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.209628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.209684 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.209688 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.209716 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.209754 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":44,"thread_start_us":80,"threads_started":1}
I20260809 16:35:03.210197 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43522 (local address 127.31.15.195:40891)
0809 16:35:03.206686 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.206690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.206698 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:03.206726 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.206727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.206740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.206879 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.206933 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208016 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.208981 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.209540 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.209975 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.210063 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.210087 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.210126 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260809 16:35:03.210533 11401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:36171 (local address 127.0.0.1:35894)
0809 16:35:03.206544 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.206552 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.206582 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.207288 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.207289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.207291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.207319 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.207320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208391 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.208392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.209469 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.209470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.210324 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.210325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.210385 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.210390 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.210417 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.210466 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":81,"threads_started":1}
I20260809 16:35:03.210736 11400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45009 (local address 127.0.0.1:39276)
0809 16:35:03.206394 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.206404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.206453 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.207126 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.207127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.207129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.207171 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.207172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208284 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.208285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.209428 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.209431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.210514 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.210515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.210586 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.210590 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.210630 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.210670 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260809 16:35:03.210973 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.31.15.194:36171)
0809 16:35:03.207110 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.207114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.207123 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:03.207160 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.207162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.207176 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.207299 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.207372 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208030 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.209550 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.210224 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.210685 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.210766 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.210855 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.210895 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260809 16:35:03.211038 11403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.206687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.31.15.193:45009)
0809 16:35:03.206982 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.206986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.206995 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:03.207030 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.207032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.207048 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.207123 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.207234 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.208220 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.209557 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.210440 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.210500 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.210895 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.210926 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.210970 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":92,"threads_started":1}
I20260809 16:35:03.210904 11252 tablet_service.cc:1467] Processing CreateTablet for tablet 44dd2975edb64f62865d94b283419d38 (DEFAULT_TABLE table=test-table [id=ac8ddad5d3524b1595d47160dca8a316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:03.211324 11252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44dd2975edb64f62865d94b283419d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:03.211649 11039 tablet_service.cc:1467] Processing CreateTablet for tablet 44dd2975edb64f62865d94b283419d38 (DEFAULT_TABLE table=test-table [id=ac8ddad5d3524b1595d47160dca8a316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:03.211681 10811 tablet_service.cc:1467] Processing CreateTablet for tablet 44dd2975edb64f62865d94b283419d38 (DEFAULT_TABLE table=test-table [id=ac8ddad5d3524b1595d47160dca8a316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:03.211982 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44dd2975edb64f62865d94b283419d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:03.211980 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44dd2975edb64f62865d94b283419d38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:03.213950 11406 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Bootstrap starting.
I20260809 16:35:03.214568 11406 tablet_bootstrap.cc:654] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:03.214782 11406 log.cc:824] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:03.215632 11406 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: No bootstrap required, opened a new log
I20260809 16:35:03.215709 11406 ts_tablet_manager.cc:1397] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:03.217495 11406 raft_consensus.cc:348] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.217621 11406 raft_consensus.cc:374] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:03.217643 11406 raft_consensus.cc:729] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281bfabfe11f4b63b3c1e1cb4c32bfcb, State: Initialized, Role: FOLLOWER
I20260809 16:35:03.217756 11406 consensus_queue.cc:260] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.218005 11406 ts_tablet_manager.cc:1428] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:35:03.218091 11366 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
I20260809 16:35:03.218514 11410 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Bootstrap starting.
I20260809 16:35:03.219215 11410 tablet_bootstrap.cc:654] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:03.219532 11410 log.cc:824] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:03.219828 11411 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Bootstrap starting.
I20260809 16:35:03.220191 11410 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: No bootstrap required, opened a new log
I20260809 16:35:03.220257 11410 ts_tablet_manager.cc:1397] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:03.220351 11411 tablet_bootstrap.cc:654] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:03.220633 11411 log.cc:824] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:03.221235 11411 tablet_bootstrap.cc:492] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: No bootstrap required, opened a new log
I20260809 16:35:03.221293 11411 ts_tablet_manager.cc:1397] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:03.221803 11410 raft_consensus.cc:348] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.221944 11410 raft_consensus.cc:374] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:03.222029 11410 raft_consensus.cc:729] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4eba99d45d4d799f5b5381e7758c81, State: Initialized, Role: FOLLOWER
I20260809 16:35:03.222154 11410 consensus_queue.cc:260] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.222407 11410 ts_tablet_manager.cc:1428] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:35:03.222493 10923 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
I20260809 16:35:03.223433 11411 raft_consensus.cc:348] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.223557 11411 raft_consensus.cc:374] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:03.223577 11411 raft_consensus.cc:729] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af9bdf77e0d472fa940ef53366c1775, State: Initialized, Role: FOLLOWER
I20260809 16:35:03.223651 11411 consensus_queue.cc:260] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.223822 11411 ts_tablet_manager.cc:1428] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:35:03.223872 11153 heartbeater.cc:503] Master 127.31.15.254:40851 was elected leader, sending a full tablet report...
W20260809 16:35:03.282310 11155 tablet.cc:2369] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:35:03.287022 11424 raft_consensus.cc:482] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:03.287132 11424 raft_consensus.cc:504] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.287485 11424 leader_election.cc:288] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009), 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:03.290303 11403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.287729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:47283 (local address 127.31.15.193:45009)
0809 16:35:03.287781 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.287783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.287791 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:03.287819 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.287820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.287829 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.287892 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.288063 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.288526 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.289606 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.289970 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.290030 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.290038 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.290179 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.290221 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:35:03.290299 11156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.287775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40891 (local address 127.31.15.194:45111)
0809 16:35:03.287816 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.287824 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.287853 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.288007 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.288008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.288010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.288083 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.288085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.288685 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.288686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.289506 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.289508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.290016 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.290017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.290097 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.290100 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.290153 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.290195 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:35:03.290311 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.287655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45009 (local address 127.31.15.194:47283)
0809 16:35:03.287709 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:03.287721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:03.287766 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:03.287904 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:03.287906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:03.287908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:03.287992 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.287994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.288593 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.288595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.289504 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:03.289507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.290034 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.290035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:03.290117 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.290121 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:03.290169 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:35:03.290206 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260809 16:35:03.290282 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.287866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:45111 (local address 127.31.15.195:40891)
0809 16:35:03.287920 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.287921 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.287927 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:03.287948 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.287948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.287955 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.287998 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:03.288141 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.288607 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.289584 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.289949 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.290012 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.290021 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.290171 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.290208 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:35:03.290795 11294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44dd2975edb64f62865d94b283419d38" candidate_uuid: "2af9bdf77e0d472fa940ef53366c1775" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" is_pre_election: true
I20260809 16:35:03.290846 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44dd2975edb64f62865d94b283419d38" candidate_uuid: "2af9bdf77e0d472fa940ef53366c1775" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd4eba99d45d4d799f5b5381e7758c81" is_pre_election: true
I20260809 16:35:03.290931 11294 raft_consensus.cc:2454] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af9bdf77e0d472fa940ef53366c1775 in term 0.
I20260809 16:35:03.290992 10844 raft_consensus.cc:2454] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2af9bdf77e0d472fa940ef53366c1775 in term 0.
I20260809 16:35:03.291172 10962 leader_election.cc:302] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281bfabfe11f4b63b3c1e1cb4c32bfcb, 2af9bdf77e0d472fa940ef53366c1775; no voters:
I20260809 16:35:03.291338 11424 raft_consensus.cc:2781] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:03.291399 11424 raft_consensus.cc:482] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:03.291417 11424 raft_consensus.cc:3037] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:03.292090 11424 raft_consensus.cc:504] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.292196 11424 leader_election.cc:288] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [CANDIDATE]: Term 1 election: Requested vote from peers fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009), 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:03.292356 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44dd2975edb64f62865d94b283419d38" candidate_uuid: "2af9bdf77e0d472fa940ef53366c1775" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd4eba99d45d4d799f5b5381e7758c81"
I20260809 16:35:03.292383 11294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44dd2975edb64f62865d94b283419d38" candidate_uuid: "2af9bdf77e0d472fa940ef53366c1775" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
I20260809 16:35:03.292429 10844 raft_consensus.cc:3037] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:03.292446 11294 raft_consensus.cc:3037] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:03.293108 11294 raft_consensus.cc:2454] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af9bdf77e0d472fa940ef53366c1775 in term 1.
I20260809 16:35:03.293150 10844 raft_consensus.cc:2454] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2af9bdf77e0d472fa940ef53366c1775 in term 1.
I20260809 16:35:03.293263 10962 leader_election.cc:302] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281bfabfe11f4b63b3c1e1cb4c32bfcb, 2af9bdf77e0d472fa940ef53366c1775; no voters:
I20260809 16:35:03.293344 11424 raft_consensus.cc:2781] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:03.293493 11424 raft_consensus.cc:686] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 LEADER]: Becoming Leader. State: Replica: 2af9bdf77e0d472fa940ef53366c1775, State: Running, Role: LEADER
I20260809 16:35:03.293576 11424 consensus_queue.cc:237] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:03.294399 9292 catalog_manager.cc:5495] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "2af9bdf77e0d472fa940ef53366c1775" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:03.305073 11404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.303098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.31.15.194:36171)
0809 16:35:03.303137 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.303138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.303147 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:03.303196 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.303197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.303212 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.303297 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:03.303383 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.303851 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.304547 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.304718 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.304762 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.304853 (+ 91us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:03.304940 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:03.304991 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.305006 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.305020 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:35:03.306630 10844 raft_consensus.cc:1261] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Refusing update from remote peer 2af9bdf77e0d472fa940ef53366c1775: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:03.306676 11294 raft_consensus.cc:1261] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Refusing update from remote peer 2af9bdf77e0d472fa940ef53366c1775: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:03.306979 11426 consensus_queue.cc:1035] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:03.307142 11424 consensus_queue.cc:1035] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [LEADER]: Connected to new peer: Peer: permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:03.310426 11429 mvcc.cc:205] Tried to move back new op lower bound from 7316657370341679104 to 7316657370290982912. Current Snapshot: MvccSnapshot[applied={T|T < 7316657370341679104}]
I20260809 16:35:03.310907 11430 mvcc.cc:205] Tried to move back new op lower bound from 7316657370341679104 to 7316657370290982912. Current Snapshot: MvccSnapshot[applied={T|T < 7316657370341679104}]
I20260809 16:35:03.312960 9293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:03.313046 9293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:03.314534 9293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af165bb537bf4043aa288845e9f93043: Sending DeleteTablet for 3 replicas of tablet 44dd2975edb64f62865d94b283419d38
I20260809 16:35:03.314841 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet 44dd2975edb64f62865d94b283419d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:03 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:39276
I20260809 16:35:03.314895 11252 tablet_service.cc:1514] Processing DeleteTablet for tablet 44dd2975edb64f62865d94b283419d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:03 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:43522
I20260809 16:35:03.315076 11444 tablet_replica.cc:330] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: stopping tablet replica
I20260809 16:35:03.315172 11444 raft_consensus.cc:2229] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:03.315313 11444 raft_consensus.cc:2258] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:03.315481 11445 tablet_replica.cc:330] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: stopping tablet replica
I20260809 16:35:03.315579 11445 raft_consensus.cc:2229] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:03.315680 11445 raft_consensus.cc:2258] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:03.315899 11444 ts_tablet_manager.cc:1905] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:03.316089 11039 tablet_service.cc:1514] Processing DeleteTablet for tablet 44dd2975edb64f62865d94b283419d38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:03 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:35894
I20260809 16:35:03.316120 11445 ts_tablet_manager.cc:1905] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:03.316134 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 9244
I20260809 16:35:03.325346 11445 ts_tablet_manager.cc:1918] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:03.325436 11445 log.cc:1196] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/44dd2975edb64f62865d94b283419d38
I20260809 16:35:03.325670 11445 ts_tablet_manager.cc:1939] T 44dd2975edb64f62865d94b283419d38 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting consensus metadata
I20260809 16:35:03.327592 11444 ts_tablet_manager.cc:1918] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:03.327695 11444 log.cc:1196] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/44dd2975edb64f62865d94b283419d38
I20260809 16:35:03.327970 11444 ts_tablet_manager.cc:1939] T 44dd2975edb64f62865d94b283419d38 P fd4eba99d45d4d799f5b5381e7758c81: Deleting consensus metadata
W20260809 16:35:03.330916 11181 connection.cc:535] server connection from 127.0.0.1:43522 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 16:35:03.331152 10728 connection.cc:535] server connection from 127.0.0.1:39276 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:35:03.335378 11447 tablet_replica.cc:330] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: stopping tablet replica
I20260809 16:35:03.335507 11447 raft_consensus.cc:2229] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:03.335650 11447 raft_consensus.cc:2258] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:03.336154 11447 ts_tablet_manager.cc:1905] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:03.336171 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.315524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.31.15.253:41239)
0809 16:35:03.315618 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.315621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.328447 (+ 12826us) server_negotiation.cc:407] Connection header received
0809 16:35:03.328504 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.328505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.328520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.328584 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.329273 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.330060 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.331059 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.331403 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.331467 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.333127 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.333129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.333129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.333136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.334890 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.335308 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.335309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.335309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.335368 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.335568 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.335568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.335569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.335696 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.335776 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.335977 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.336074 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 16:35:03.337744 11447 ts_tablet_manager.cc:1918] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:03.337822 11447 log.cc:1196] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/44dd2975edb64f62865d94b283419d38
I20260809 16:35:03.338117 11447 ts_tablet_manager.cc:1939] T 44dd2975edb64f62865d94b283419d38 P 2af9bdf77e0d472fa940ef53366c1775: Deleting consensus metadata
W20260809 16:35:03.338441 10966 connection.cc:441] server connection from 127.0.0.1:35894 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35894 (request call id 1) could send its response
I20260809 16:35:03.341109 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.332913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.31.15.252:37577)
0809 16:35:03.332955 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.332957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.333516 (+ 559us) server_negotiation.cc:407] Connection header received
0809 16:35:03.333554 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.333555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.333567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.333616 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.334639 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.335359 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.336698 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.336971 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.337009 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.338398 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.338399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.338399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.338406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.339923 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.340291 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.340292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.340292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.340353 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.340564 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.340565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.340565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.340688 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.340761 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.340954 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.341021 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:35:03.341216 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.332966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.31.15.253:41239)
0809 16:35:03.333006 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.333007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.333433 (+ 426us) server_negotiation.cc:407] Connection header received
0809 16:35:03.333461 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.333462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.333472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.333521 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.334056 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.334777 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.335974 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.336226 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.336269 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.337991 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.337993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.337993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.338000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.340061 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.340424 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.340425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.340425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.340475 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.340672 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.340673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.340674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.340788 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.340857 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.341051 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.341125 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:35:03.343442 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:03.315577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53452 (local address 127.31.15.252:37577)
0809 16:35:03.315650 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:03.315652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:03.337091 (+ 21439us) server_negotiation.cc:407] Connection header received
0809 16:35:03.337173 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:03.337174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:03.337184 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:03.337211 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:03.337980 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.338644 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.339549 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:03.339826 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:03.339865 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:03.341091 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:03.341092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:03.341092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:03.341098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:03.342466 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.342815 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.342815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.342816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.342862 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:03.343034 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:03.343034 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:03.343034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:03.343116 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:03.343187 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:03.343319 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:35:03.343388 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 16:35:04.311161 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.310818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:48611)
0809 16:35:04.311007 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.311100 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":66,"threads_started":1}
W20260809 16:35:04.311601 11153 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:40851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.311707 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.310850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:43595)
0809 16:35:04.311596 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.311659 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":70,"threads_started":1}
I20260809 16:35:04.311865 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.311172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:53165)
0809 16:35:04.311561 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.311659 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":46,"threads_started":1}
W20260809 16:35:04.311935 10923 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:40851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
W20260809 16:35:04.312107 11366 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:40851 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.815089 11464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af165bb537bf4043aa288845e9f93043)
I20260809 16:35:04.815259 11465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af165bb537bf4043aa288845e9f93043)
I20260809 16:35:04.815258 11464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:04.815418 11465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:04.815587 11464 leader_election.cc:288] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252:37577)
I20260809 16:35:04.815970 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.815715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43522)
0809 16:35:04.815871 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.815925 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":48,"threads_started":1}
I20260809 16:35:04.815970 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.815715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43514)
0809 16:35:04.815875 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.815933 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":45,"threads_started":1}
W20260809 16:35:04.816089 10283 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
W20260809 16:35:04.816092 10603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.816553 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.816433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43530)
0809 16:35:04.816494 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.816525 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 16:35:04.816639 11465 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 748c98777b13417c8a08a0828950f29c (127.31.15.253:41239)
W20260809 16:35:04.816747 10603 leader_election.cc:334] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.816905 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.816821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43534)
0809 16:35:04.816856 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.816878 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:35:04.816982 10283 leader_election.cc:334] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.819231 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.815896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:41239 (local address 127.0.0.1:50326)
0809 16:35:04.816031 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.816041 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:04.816098 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:04.816481 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:04.816482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:04.816484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:04.816556 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:04.816559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.817371 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.817372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:04.818332 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:04.818335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.818998 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.818999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:04.819075 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:04.819082 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:35:04.819129 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:35:04.819170 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":54,"threads_started":1}
I20260809 16:35:04.819442 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.816218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.31.15.253:41239)
0809 16:35:04.816357 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:04.816358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:04.816365 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:04.816386 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:04.816386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:04.816396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:04.816621 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:04.816656 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.817294 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.818413 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.818917 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.818996 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:04.819008 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:04.819329 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 16:35:04.819374 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 16:35:04.819578 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.815987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.31.15.252:37577)
0809 16:35:04.816312 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:04.816314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:04.816322 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:04.816348 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:04.816349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:04.816359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:04.816431 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:04.816915 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.817619 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.818634 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.819165 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.819219 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:04.819228 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:04.819483 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 16:35:04.819524 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":32,"threads_started":1}
I20260809 16:35:04.819710 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.815746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:37577 (local address 127.0.0.1:53468)
0809 16:35:04.815892 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.815904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:04.815962 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:04.816508 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:04.816509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:04.816512 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:04.816566 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:04.816568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.817698 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.817699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:04.818561 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:04.818564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:04.819257 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:04.819258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:04.819335 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:04.819339 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:04.819388 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:35:04.819433 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":51,"threads_started":1}
I20260809 16:35:04.819906 10320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "748c98777b13417c8a08a0828950f29c" is_pre_election: true
I20260809 16:35:04.819906 10642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "748c98777b13417c8a08a0828950f29c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" is_pre_election: true
I20260809 16:35:04.820007 10320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a7ee897e0ca48ea87575b1bb522f45c in term 1.
I20260809 16:35:04.820017 10642 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 748c98777b13417c8a08a0828950f29c in term 1.
I20260809 16:35:04.820194 10279 leader_election.cc:302] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 748c98777b13417c8a08a0828950f29c, 7a7ee897e0ca48ea87575b1bb522f45c; no voters: af165bb537bf4043aa288845e9f93043
I20260809 16:35:04.820184 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 748c98777b13417c8a08a0828950f29c, 7a7ee897e0ca48ea87575b1bb522f45c; no voters: af165bb537bf4043aa288845e9f93043
I20260809 16:35:04.820297 11464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:35:04.820307 11465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 16:35:04.820336 11465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Starting leader election (detected failure of leader af165bb537bf4043aa288845e9f93043)
I20260809 16:35:04.820338 11464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Starting leader election (detected failure of leader af165bb537bf4043aa288845e9f93043)
I20260809 16:35:04.820354 11465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:35:04.820358 11464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:35:04.821055 11464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:04.821054 11465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:04.821174 11464 leader_election.cc:288] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 election: Requested vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 7a7ee897e0ca48ea87575b1bb522f45c (127.31.15.252:37577)
I20260809 16:35:04.821331 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.821235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43546)
0809 16:35:04.821279 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.821308 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:35:04.821332 10642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "748c98777b13417c8a08a0828950f29c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7a7ee897e0ca48ea87575b1bb522f45c"
I20260809 16:35:04.821457 10642 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 748c98777b13417c8a08a0828950f29c in current term 2: Already voted for candidate 7a7ee897e0ca48ea87575b1bb522f45c in this term.
I20260809 16:35:04.821445 10320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "748c98777b13417c8a08a0828950f29c"
I20260809 16:35:04.821517 10320 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7a7ee897e0ca48ea87575b1bb522f45c in current term 2: Already voted for candidate 748c98777b13417c8a08a0828950f29c in this term.
I20260809 16:35:04.821380 11465 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 election: Requested vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 748c98777b13417c8a08a0828950f29c (127.31.15.253:41239)
I20260809 16:35:04.821627 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.821522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43562)
0809 16:35:04.821563 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.821589 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:35:04.821741 10603 leader_election.cc:334] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.821750 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.821293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43558)
0809 16:35:04.821357 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.821727 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:35:04.821784 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7a7ee897e0ca48ea87575b1bb522f45c; no voters: 748c98777b13417c8a08a0828950f29c, af165bb537bf4043aa288845e9f93043
I20260809 16:35:04.821875 11465 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 16:35:04.822026 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:04.821950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43566)
0809 16:35:04.821986 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:04.822012 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:35:04.822088 10283 leader_election.cc:334] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:04.822160 10283 leader_election.cc:302] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 748c98777b13417c8a08a0828950f29c; no voters: 7a7ee897e0ca48ea87575b1bb522f45c, af165bb537bf4043aa288845e9f93043
I20260809 16:35:04.822233 11464 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 16:35:05.312269 11480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:05.311941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:43339)
0809 16:35:05.312098 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:05.312143 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260809 16:35:05.312620 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:05.312282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:51895)
0809 16:35:05.312504 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:05.312573 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
I20260809 16:35:05.312839 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:05.312518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:56103)
0809 16:35:05.312762 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:05.312800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":61,"threads_started":1}
I20260809 16:35:06.313074 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.312778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:34103)
0809 16:35:06.312918 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.312960 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260809 16:35:06.313441 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.313107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:41633)
0809 16:35:06.313360 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.313397 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":54,"threads_started":1}
I20260809 16:35:06.313761 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.313565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:60825)
0809 16:35:06.313693 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.313725 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260809 16:35:06.371753 11490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:06.371867 11490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:06.372048 11490 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 748c98777b13417c8a08a0828950f29c (127.31.15.253:41239)
I20260809 16:35:06.372429 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.372180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43770)
0809 16:35:06.372340 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.372378 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":49,"threads_started":1}
I20260809 16:35:06.372429 10320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "748c98777b13417c8a08a0828950f29c" is_pre_election: true
I20260809 16:35:06.372527 10320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a7ee897e0ca48ea87575b1bb522f45c in term 2.
I20260809 16:35:06.372707 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 748c98777b13417c8a08a0828950f29c, 7a7ee897e0ca48ea87575b1bb522f45c; no voters:
I20260809 16:35:06.372828 11490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 16:35:06.372941 11490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:06.373016 11490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:35:06.374146 11490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:06.374389 11490 leader_election.cc:288] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 election: Requested vote from peers af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851), 748c98777b13417c8a08a0828950f29c (127.31.15.253:41239)
I20260809 16:35:06.375365 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.375238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43776)
0809 16:35:06.375301 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.375337 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:35:06.375414 10320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "748c98777b13417c8a08a0828950f29c"
I20260809 16:35:06.375495 10320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:35:06.376211 10320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a7ee897e0ca48ea87575b1bb522f45c in term 3.
W20260809 16:35:06.376307 10603 leader_election.cc:334] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:06.376399 10603 leader_election.cc:302] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 748c98777b13417c8a08a0828950f29c, 7a7ee897e0ca48ea87575b1bb522f45c; no voters:
I20260809 16:35:06.376477 11490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:35:06.376580 11490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [term 3 LEADER]: Becoming Leader. State: Replica: 7a7ee897e0ca48ea87575b1bb522f45c, State: Running, Role: LEADER
I20260809 16:35:06.376632 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.376509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43778)
0809 16:35:06.376557 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.376581 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:35:06.376674 10603 leader_election.cc:334] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
I20260809 16:35:06.376659 11490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } }
I20260809 16:35:06.377012 11494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a7ee897e0ca48ea87575b1bb522f45c. Latest consensus state: current_term: 3 leader_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:06.377197 11494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:06.377406 11496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:06.377919 11496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:06.377995 11496 catalog_manager.cc:1259] Loaded cluster ID: e85e7a17a4e440bca1d5638ce5796862
I20260809 16:35:06.378026 11496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:06.378238 11496 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:06.378321 11496 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Loaded TSK: 0
I20260809 16:35:06.378512 11496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:35:06.419056 10625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:06.420794 10625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:06.421833 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.421722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43786)
0809 16:35:06.421762 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.421800 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:35:06.421952 10320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [term 3 FOLLOWER]: Refusing update from remote peer 7a7ee897e0ca48ea87575b1bb522f45c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 16:35:06.422155 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.422074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.0.0.1:43800)
0809 16:35:06.422103 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.422130 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 16:35:06.422188 11490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [LEADER]: Connected to new peer: Peer: permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 16:35:06.422315 10603 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c -> Peer af165bb537bf4043aa288845e9f93043 (127.31.15.254:40851): Couldn't send request to peer af165bb537bf4043aa288845e9f93043. Status: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 16:35:06.422746 11499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a7ee897e0ca48ea87575b1bb522f45c. Latest consensus state: current_term: 3 leader_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:06.422837 11499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:06.423311 11490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:06.423403 11490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:06.423697 11499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "af165bb537bf4043aa288845e9f93043" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40851 } } peers { permanent_uuid: "748c98777b13417c8a08a0828950f29c" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 41239 } attrs { promote: false } } peers { permanent_uuid: "7a7ee897e0ca48ea87575b1bb522f45c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 37577 } attrs { promote: false } } }
I20260809 16:35:06.423779 11499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748c98777b13417c8a08a0828950f29c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:35:06.428025 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.424966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45009 (local address 127.0.0.1:39288)
0809 16:35:06.425009 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.425019 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:06.425076 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:06.425311 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:06.425312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:06.425314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:06.425366 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.425368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.426137 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.426138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.427016 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.427018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.427808 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.427809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.427871 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.427875 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:06.427922 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:35:06.427958 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:35:06.428309 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.425042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.31.15.193:45009)
0809 16:35:06.425181 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.425183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.425192 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:06.425217 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.425218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.425228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.425306 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:06.425460 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.426059 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.427091 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.427689 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.427730 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.427741 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.428194 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.428241 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260809 16:35:06.428581 10811 tablet_service.cc:1467] Processing CreateTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 (DEFAULT_TABLE table=test-table [id=58fda48ce63e48338da933c0ad72dab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:06.428727 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edc67f8a00314ffb936be5c07f5b2a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:06.428887 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.425357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.31.15.195:40891)
0809 16:35:06.425716 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.425718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.426182 (+ 464us) server_negotiation.cc:407] Connection header received
0809 16:35:06.426218 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.426219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.426231 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.426294 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:06.426389 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.426854 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428079 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.428565 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428637 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.428648 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.428779 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.428821 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":64,"threads_started":1}
I20260809 16:35:06.428889 11509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.425194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40891 (local address 127.0.0.1:43524)
0809 16:35:06.425369 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.426126 (+ 757us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:06.426175 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:06.426295 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:06.426296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:06.426298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:06.426342 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.426344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.426920 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.426921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.428005 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.428009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.428646 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.428706 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.428710 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:06.428766 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:35:06.428801 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":65,"threads_started":1}
I20260809 16:35:06.429175 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.425055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:36171 (local address 127.0.0.1:35910)
0809 16:35:06.425311 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.425322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:06.425371 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:06.425553 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:06.425554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:06.425555 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:06.425592 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.425594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.427099 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.427100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.428021 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.428023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.428967 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.429023 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.429025 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:06.429069 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:35:06.429103 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":67,"threads_started":1}
I20260809 16:35:06.429198 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.425116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35910 (local address 127.31.15.194:36171)
0809 16:35:06.425227 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.425229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.425434 (+ 205us) server_negotiation.cc:407] Connection header received
0809 16:35:06.425461 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.425462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.425472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.425538 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:06.425644 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.426265 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428399 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.428900 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.428963 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.428975 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.429082 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.429135 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260809 16:35:06.429934 11516 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: Bootstrap starting.
I20260809 16:35:06.430130 11039 tablet_service.cc:1467] Processing CreateTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 (DEFAULT_TABLE table=test-table [id=58fda48ce63e48338da933c0ad72dab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:06.430141 11252 tablet_service.cc:1467] Processing CreateTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 (DEFAULT_TABLE table=test-table [id=58fda48ce63e48338da933c0ad72dab5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:06.430249 11039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edc67f8a00314ffb936be5c07f5b2a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:06.430253 11252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edc67f8a00314ffb936be5c07f5b2a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:06.430415 11516 tablet_bootstrap.cc:654] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:06.430972 11516 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: No bootstrap required, opened a new log
I20260809 16:35:06.431021 11516 ts_tablet_manager.cc:1397] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:06.431197 11516 raft_consensus.cc:348] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.431264 11516 raft_consensus.cc:374] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:06.431335 11516 raft_consensus.cc:729] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4eba99d45d4d799f5b5381e7758c81, State: Initialized, Role: FOLLOWER
I20260809 16:35:06.431406 11516 consensus_queue.cc:260] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.431589 11518 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Bootstrap starting.
I20260809 16:35:06.431722 11516 ts_tablet_manager.cc:1428] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:35:06.432080 11518 tablet_bootstrap.cc:654] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:06.432145 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.432055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:59383)
0809 16:35:06.432097 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.432121 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:35:06.432442 10922 heartbeater.cc:503] Master 127.31.15.252:37577 was elected leader, sending a full tablet report...
I20260809 16:35:06.432628 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.432548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:42275)
0809 16:35:06.432583 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.432605 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 16:35:06.432853 11518 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: No bootstrap required, opened a new log
I20260809 16:35:06.432901 11518 ts_tablet_manager.cc:1397] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:06.433009 11518 raft_consensus.cc:348] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.433061 11518 raft_consensus.cc:374] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:06.433079 11518 raft_consensus.cc:729] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281bfabfe11f4b63b3c1e1cb4c32bfcb, State: Initialized, Role: FOLLOWER
I20260809 16:35:06.433118 11518 consensus_queue.cc:260] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.433290 11518 ts_tablet_manager.cc:1428] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:06.433542 11521 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Bootstrap starting.
I20260809 16:35:06.433595 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.433460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:34431)
0809 16:35:06.433543 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.433569 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 16:35:06.433677 11365 heartbeater.cc:503] Master 127.31.15.252:37577 was elected leader, sending a full tablet report...
I20260809 16:35:06.433907 11521 tablet_bootstrap.cc:654] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:06.434381 11521 tablet_bootstrap.cc:492] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: No bootstrap required, opened a new log
I20260809 16:35:06.434422 11521 ts_tablet_manager.cc:1397] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:06.434517 11521 raft_consensus.cc:348] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.434568 11521 raft_consensus.cc:374] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:06.434585 11521 raft_consensus.cc:729] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af9bdf77e0d472fa940ef53366c1775, State: Initialized, Role: FOLLOWER
I20260809 16:35:06.434625 11521 consensus_queue.cc:260] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.434777 11521 ts_tablet_manager.cc:1428] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:06.435043 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.434950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:52643)
0809 16:35:06.434987 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.435018 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 16:35:06.435189 11149 heartbeater.cc:503] Master 127.31.15.252:37577 was elected leader, sending a full tablet report...
I20260809 16:35:06.435446 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.435221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:37173)
0809 16:35:06.435397 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.435423 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260809 16:35:06.629089 10925 tablet.cc:2369] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:35:06.681854 11370 tablet.cc:2369] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:35:06.705307 11520 raft_consensus.cc:482] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:06.705437 11520 raft_consensus.cc:504] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.705821 11520 leader_election.cc:288] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171), 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:06.708572 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.705983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:36171 (local address 127.31.15.193:46935)
0809 16:35:06.706047 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.706061 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:06.706137 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:06.706279 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:06.706281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:06.706283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:06.706359 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.706361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.706988 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.706989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.707800 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.707803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.708320 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.708321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.708387 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.708391 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:06.708443 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:35:06.708485 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:35:06.708557 11510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.706073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:46935 (local address 127.31.15.194:36171)
0809 16:35:06.706126 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.706128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.706139 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:35:06.706216 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.706217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.706226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.706272 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:06.706422 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.706902 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.707881 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.708245 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.708310 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.708321 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.708451 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.708489 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:35:06.708827 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.706092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40891 (local address 127.31.15.193:48181)
0809 16:35:06.706281 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.706290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:06.706344 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:06.706527 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:06.706529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:06.706530 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:06.706608 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.706611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.707234 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.707235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.708020 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:06.708022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.708571 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.708572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:06.708647 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.708652 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:35:06.708708 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:35:06.708749 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":122,"threads_started":1}
I20260809 16:35:06.708808 11068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc67f8a00314ffb936be5c07f5b2a19" candidate_uuid: "fd4eba99d45d4d799f5b5381e7758c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af9bdf77e0d472fa940ef53366c1775" is_pre_election: true
I20260809 16:35:06.708942 11068 raft_consensus.cc:2454] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4eba99d45d4d799f5b5381e7758c81 in term 0.
I20260809 16:35:06.709138 10728 leader_election.cc:302] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2af9bdf77e0d472fa940ef53366c1775, fd4eba99d45d4d799f5b5381e7758c81; no voters:
I20260809 16:35:06.709156 11512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.706193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:48181 (local address 127.31.15.195:40891)
0809 16:35:06.706240 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.706242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.706348 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:35:06.706450 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.706451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.706461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.706519 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:06.706673 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.707148 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.708113 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.708477 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.708548 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.708558 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.708769 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.708812 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:35:06.709252 11520 raft_consensus.cc:2781] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:06.709282 11520 raft_consensus.cc:482] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:06.709299 11520 raft_consensus.cc:3037] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:06.709358 11294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc67f8a00314ffb936be5c07f5b2a19" candidate_uuid: "fd4eba99d45d4d799f5b5381e7758c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" is_pre_election: true
I20260809 16:35:06.709461 11294 raft_consensus.cc:2454] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd4eba99d45d4d799f5b5381e7758c81 in term 0.
I20260809 16:35:06.710062 11520 raft_consensus.cc:504] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.710173 11520 leader_election.cc:288] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [CANDIDATE]: Term 1 election: Requested vote from peers 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171), 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891)
I20260809 16:35:06.710358 11294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc67f8a00314ffb936be5c07f5b2a19" candidate_uuid: "fd4eba99d45d4d799f5b5381e7758c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb"
I20260809 16:35:06.710368 11068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edc67f8a00314ffb936be5c07f5b2a19" candidate_uuid: "fd4eba99d45d4d799f5b5381e7758c81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af9bdf77e0d472fa940ef53366c1775"
I20260809 16:35:06.710439 11068 raft_consensus.cc:3037] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:06.710439 11294 raft_consensus.cc:3037] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:06.711172 11294 raft_consensus.cc:2454] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4eba99d45d4d799f5b5381e7758c81 in term 1.
I20260809 16:35:06.711184 11068 raft_consensus.cc:2454] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd4eba99d45d4d799f5b5381e7758c81 in term 1.
I20260809 16:35:06.711354 10728 leader_election.cc:302] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 281bfabfe11f4b63b3c1e1cb4c32bfcb, fd4eba99d45d4d799f5b5381e7758c81; no voters:
I20260809 16:35:06.711459 11520 raft_consensus.cc:2781] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:06.711606 11520 raft_consensus.cc:686] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 LEADER]: Becoming Leader. State: Replica: fd4eba99d45d4d799f5b5381e7758c81, State: Running, Role: LEADER
I20260809 16:35:06.711679 11520 consensus_queue.cc:237] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } }
I20260809 16:35:06.712205 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.712109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:45217)
0809 16:35:06.712150 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.712179 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:35:06.712564 10625 catalog_manager.cc:5495] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "fd4eba99d45d4d799f5b5381e7758c81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd4eba99d45d4d799f5b5381e7758c81" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:06.779332 11508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.776883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.31.15.193:45009)
0809 16:35:06.776941 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:06.776943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:06.776952 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:06.777020 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:06.777022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:06.777043 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:06.777115 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:06.777257 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.777784 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.778550 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:06.778744 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:06.778808 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:06.778967 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:06.779073 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:06.779145 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:06.779197 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:35:06.779220 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:35:06.780449 11294 raft_consensus.cc:1261] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Refusing update from remote peer fd4eba99d45d4d799f5b5381e7758c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:06.780637 11068 raft_consensus.cc:1261] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Refusing update from remote peer fd4eba99d45d4d799f5b5381e7758c81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:06.780755 11520 consensus_queue.cc:1035] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "281bfabfe11f4b63b3c1e1cb4c32bfcb" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:06.780930 11534 consensus_queue.cc:1035] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2af9bdf77e0d472fa940ef53366c1775" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 36171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:06.781690 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.781567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:38315)
0809 16:35:06.781620 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.781655 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 16:35:06.781826 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.781689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:37505)
0809 16:35:06.781766 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.781793 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 16:35:06.783202 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.782932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:59687)
0809 16:35:06.783136 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.783171 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260809 16:35:06.783367 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.783128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.194:59719)
0809 16:35:06.783312 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.783342 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 16:35:06.783898 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.783805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.193:53173)
0809 16:35:06.783835 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.783863 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 16:35:06.784142 11489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.784026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40851 (local address 127.31.15.195:43887)
0809 16:35:06.784081 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.784115 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:40851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:35:06.785665 10625 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:06.785760 10625 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:06.787035 10625 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a7ee897e0ca48ea87575b1bb522f45c: Sending DeleteTablet for 3 replicas of tablet edc67f8a00314ffb936be5c07f5b2a19
I20260809 16:35:06.787427 11252 tablet_service.cc:1514] Processing DeleteTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:06 UTC) from {username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:43524
I20260809 16:35:06.787448 10811 tablet_service.cc:1514] Processing DeleteTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:06 UTC) from {username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:39288
I20260809 16:35:06.787505 11039 tablet_service.cc:1514] Processing DeleteTablet for tablet edc67f8a00314ffb936be5c07f5b2a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:06 UTC) from {username='oryx', principal='oryx/127.31.15.252@KRBTEST.COM'} at 127.0.0.1:35910
I20260809 16:35:06.787776 11555 tablet_replica.cc:330] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81: stopping tablet replica
I20260809 16:35:06.787839 11555 raft_consensus.cc:2229] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:06.787912 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 10543
I20260809 16:35:06.787935 11555 raft_consensus.cc:2258] T edc67f8a00314ffb936be5c07f5b2a19 P fd4eba99d45d4d799f5b5381e7758c81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:06.788455 11553 tablet_replica.cc:330] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: stopping tablet replica
I20260809 16:35:06.788514 11553 raft_consensus.cc:2229] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:06.788555 11553 raft_consensus.cc:2258] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:06.793392 11556 tablet_replica.cc:330] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: stopping tablet replica
I20260809 16:35:06.793457 11556 raft_consensus.cc:2229] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:06.793519 11556 raft_consensus.cc:2258] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:06.795809 11553 ts_tablet_manager.cc:1905] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:06.797722 11553 ts_tablet_manager.cc:1918] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:06.797807 11553 log.cc:1196] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/edc67f8a00314ffb936be5c07f5b2a19
I20260809 16:35:06.798123 11553 ts_tablet_manager.cc:1939] T edc67f8a00314ffb936be5c07f5b2a19 P 281bfabfe11f4b63b3c1e1cb4c32bfcb: Deleting consensus metadata
I20260809 16:35:06.798398 11556 ts_tablet_manager.cc:1905] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:06.798501 10599 catalog_manager.cc:4841] TS 281bfabfe11f4b63b3c1e1cb4c32bfcb (127.31.15.195:40891): tablet edc67f8a00314ffb936be5c07f5b2a19 (table test-table [id=58fda48ce63e48338da933c0ad72dab5]) successfully deleted
I20260809 16:35:06.799732 11556 ts_tablet_manager.cc:1918] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:06.799793 11556 log.cc:1196] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/edc67f8a00314ffb936be5c07f5b2a19
I20260809 16:35:06.800001 11556 ts_tablet_manager.cc:1939] T edc67f8a00314ffb936be5c07f5b2a19 P 2af9bdf77e0d472fa940ef53366c1775: Deleting consensus metadata
I20260809 16:35:06.800318 10599 catalog_manager.cc:4841] TS 2af9bdf77e0d472fa940ef53366c1775 (127.31.15.194:36171): tablet edc67f8a00314ffb936be5c07f5b2a19 (table test-table [id=58fda48ce63e48338da933c0ad72dab5]) successfully deleted
I20260809 16:35:06.802758 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 10936
I20260809 16:35:06.802821 11511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:06.802696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45009 (local address 127.0.0.1:55478)
0809 16:35:06.802758 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:06.802788 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 16:35:06.806002 10599 catalog_manager.cc:4584] TS fd4eba99d45d4d799f5b5381e7758c81 (127.31.15.193:45009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet edc67f8a00314ffb936be5c07f5b2a19: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45009: connect: Connection refused (error 111)
I20260809 16:35:06.814054 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 11159
I20260809 16:35:06.824283 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 10079
I20260809 16:35:06.833315 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 10389
2026-08-09T16:35:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7980 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:35:06 dist-test-slave-21b9 krb5kdc[11561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:35:06 dist-test-slave-21b9 krb5kdc[11561](info): set up 2 sockets
Aug 09 16:35:06 dist-test-slave-21b9 krb5kdc[11561](info): commencing operation
krb5kdc: starting...
W20260809 16:35:07.984282 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab.
Entry for principal oryx/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab.
I20260809 16:35:08.105609 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36519
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab
--principal=oryx/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:36519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:08.186101 11582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:08.186296 11582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:08.186316 11582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:08.187484 11582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:08.187526 11582 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 16:35:08.187547 11582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:08.187559 11582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:08.187569 11582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:08.187580 11582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:08.189082 11582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:36519
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.254.keytab
--principal=oryx/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:08.189329 11582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:08.189580 11582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:08.194139 11589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.194132 11590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.194134 11593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.194216 11582 server_base.cc:1034] running on GCE node
I20260809 16:35:08.194589 11582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:08.194864 11582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:08.196003 11582 hybrid_clock.cc:648] HybridClock initialized: now 1786293308195988 us; error 37 us; skew 500 ppm
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:08.198387 11582 init.cc:377] Logged in from keytab as oryx/127.31.15.254@KRBTEST.COM (short username oryx)
I20260809 16:35:08.198799 11582 webserver.cc:460] Webserver started at http://127.31.15.254:46157/ using document root <none> and password file <none>
I20260809 16:35:08.198999 11582 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:08.199045 11582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:08.199136 11582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:08.200199 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "82ff5bdc6ff4449c946026e32daf2475"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "1358a5c9ae2b48d72527995d011e2962"
server_key_iv: "512dd07c2ae06678b41dba39c1ce8343"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.200495 11582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "82ff5bdc6ff4449c946026e32daf2475"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "1358a5c9ae2b48d72527995d011e2962"
server_key_iv: "512dd07c2ae06678b41dba39c1ce8343"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.201696 11582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:08.202319 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.202467 11582 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:08.202544 11582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "82ff5bdc6ff4449c946026e32daf2475"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "1358a5c9ae2b48d72527995d011e2962"
server_key_iv: "512dd07c2ae06678b41dba39c1ce8343"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.202610 11582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:08.212766 11582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:08.214705 11582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:08.214823 11582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:08.219332 11582 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:36519
I20260809 16:35:08.219403 11703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:36519 every 8 connection(s)
I20260809 16:35:08.219686 11582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:08.219962 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 11582
I20260809 16:35:08.220067 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:08.220274 31807 external_mini_cluster.cc:1428] Setting key 39728fe3840162fd0f0db3772b340348
I20260809 16:35:08.220407 11706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.223621 11706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Bootstrap starting.
I20260809 16:35:08.224218 11706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.224506 11706 log.cc:824] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:08.225149 11706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: No bootstrap required, opened a new log
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:08.227240 11706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } }
I20260809 16:35:08.227407 11706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.227455 11706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ff5bdc6ff4449c946026e32daf2475, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.227586 11706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } }
I20260809 16:35:08.227664 11706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:08.227695 11706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:08.227766 11706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:08.228327 11706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } }
I20260809 16:35:08.228441 11706 leader_election.cc:302] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82ff5bdc6ff4449c946026e32daf2475; no voters:
I20260809 16:35:08.228619 11706 leader_election.cc:288] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:08.228660 11715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:08.228786 11715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [term 1 LEADER]: Becoming Leader. State: Replica: 82ff5bdc6ff4449c946026e32daf2475, State: Running, Role: LEADER
I20260809 16:35:08.228891 11706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:08.228885 11715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } }
I20260809 16:35:08.230531 11716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82ff5bdc6ff4449c946026e32daf2475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } } }
I20260809 16:35:08.230782 11716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:08.230657 11717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82ff5bdc6ff4449c946026e32daf2475. Latest consensus state: current_term: 1 leader_uuid: "82ff5bdc6ff4449c946026e32daf2475" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82ff5bdc6ff4449c946026e32daf2475" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36519 } } }
I20260809 16:35:08.230953 11717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475 [sys.catalog]: This master's current role is: LEADER
W20260809 16:35:08.231204 11737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:35:08.231246 11737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:35:08.231329 11739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:08.231480 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.220748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59348 (local address 127.31.15.254:36519)
0809 16:35:08.220938 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.220948 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.220982 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:35:08.221495 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.221504 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.221567 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.221715 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.222311 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.223718 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.224508 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.224986 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.225086 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.227478 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.227483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.227483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.227496 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.229493 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.229828 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.229831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.229832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.229947 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.230169 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.230170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.230171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.230485 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.230636 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.231032 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.231181 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
I20260809 16:35:08.231532 11739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:08.232903 11739 catalog_manager.cc:1347] Generated new cluster ID: 4d02ea1f7b9a466f8a9b06f812e88aa2
I20260809 16:35:08.233088 11739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:08.253548 11739 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:08.254056 11739 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:08.258766 11739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Generated new TSK 0
I20260809 16:35:08.258944 11739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
I20260809 16:35:08.293787 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36519
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:08.401355 11747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:08.401548 11747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:08.401571 11747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:08.402648 11747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:08.402683 11747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:08.402701 11747 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:35:08.402717 11747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:08.404170 11747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:36519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:08.404351 11747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:08.404531 11747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:08.409322 11760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.409389 11747 server_base.cc:1034] running on GCE node
W20260809 16:35:08.409340 11758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.409332 11757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.409768 11747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:08.409992 11747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:08.411130 11747 hybrid_clock.cc:648] HybridClock initialized: now 1786293308411118 us; error 28 us; skew 500 ppm
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:08.413511 11747 init.cc:377] Logged in from keytab as oryx/127.31.15.193@KRBTEST.COM (short username oryx)
I20260809 16:35:08.413901 11747 webserver.cc:460] Webserver started at http://127.31.15.193:43649/ using document root <none> and password file <none>
I20260809 16:35:08.414101 11747 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:08.414149 11747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:08.414276 11747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:08.415308 11747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "17985b0360fe40258cb28f8796082519"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "9d26a00aa397c5886b4fb13d5f9a19be"
server_key_iv: "7dde77cb7b4762d5b6003eb05a501cb7"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.415650 11747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "17985b0360fe40258cb28f8796082519"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "9d26a00aa397c5886b4fb13d5f9a19be"
server_key_iv: "7dde77cb7b4762d5b6003eb05a501cb7"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.416883 11747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:08.417544 11769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.417691 11747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:08.417773 11747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "17985b0360fe40258cb28f8796082519"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "9d26a00aa397c5886b4fb13d5f9a19be"
server_key_iv: "7dde77cb7b4762d5b6003eb05a501cb7"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.417838 11747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:08.428638 11747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:08.430794 11747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:08.430922 11747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:08.431427 11747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:08.431468 11747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.431488 11747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:08.431500 11747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.437629 11747 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:43145
I20260809 16:35:08.437695 11940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:43145 every 8 connection(s)
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:08.438006 11747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:35:08.439662 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 11747
I20260809 16:35:08.439785 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:08.439999 31807 external_mini_cluster.cc:1428] Setting key b70c8a2089bdefa241659b1775b03394
I20260809 16:35:08.441022 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.431819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:59251 (local address 127.31.15.254:36519)
0809 16:35:08.431880 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.431882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.432045 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:35:08.432507 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.432509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.432521 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.432597 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.433341 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.434090 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.435554 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.435865 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.435947 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.438289 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.438291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.438291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.438300 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.439673 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.440079 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.440080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.440081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.440148 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.440403 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.440403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.440403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.440505 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.440565 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.440853 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.440940 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:35:08.441083 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.431788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.193:59251)
0809 16:35:08.431923 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.431962 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.432420 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.432667 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.432671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.432965 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.433243 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.433250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.434199 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.434201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.435451 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.435455 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.435951 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.435952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.436067 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.436520 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.436529 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.438202 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.439754 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.439759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.439761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.439996 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.440210 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.440212 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.440212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.440341 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.440561 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.440566 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.440723 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.440811 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.440929 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":71,"threads_started":1}
I20260809 16:35:08.441686 11942 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36519
I20260809 16:35:08.441771 11942 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:08.441982 11942 heartbeater.cc:511] Master 127.31.15.254:36519 requested a full tablet report, sending...
I20260809 16:35:08.442579 11629 ts_manager.cc:194] Registered new tserver with Master: 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145)
I20260809 16:35:08.443503 11629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:59251
I20260809 16:35:08.448076 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.442513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:38399 (local address 127.31.15.254:36519)
0809 16:35:08.442553 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.442555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.442678 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:35:08.442766 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.442766 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.442776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.442802 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.443210 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.443864 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.444756 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.444933 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.444986 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.445872 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.445873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.445873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.445879 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.447082 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.447477 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.447478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.447478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.447522 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.447694 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.447694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.447695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.447775 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.447817 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.447966 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.448018 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:35:08.448143 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.442457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.193:38399)
0809 16:35:08.442618 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.442628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.442701 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.442882 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.442882 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.443079 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.443133 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.443135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.443981 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.443981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.444673 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.444676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.444998 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.444999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.445069 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.445431 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.445436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.445782 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.447169 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.447170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.447170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.447406 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.447575 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.447576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.447576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.447641 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.447823 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.447824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.447870 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.447952 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.448047 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
I20260809 16:35:08.465893 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36519
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:08.545493 11954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:08.545687 11954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:08.545718 11954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:08.546835 11954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:08.546875 11954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:08.546901 11954 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:35:08.546924 11954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:08.548429 11954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:36519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:08.548666 11954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:08.548869 11954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:08.554260 11963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.554328 11964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.554374 11968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.554272 11954 server_base.cc:1034] running on GCE node
I20260809 16:35:08.554597 11954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:08.554814 11954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:08.555956 11954 hybrid_clock.cc:648] HybridClock initialized: now 1786293308555943 us; error 33 us; skew 500 ppm
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:08.558346 11954 init.cc:377] Logged in from keytab as oryx/127.31.15.194@KRBTEST.COM (short username oryx)
I20260809 16:35:08.558784 11954 webserver.cc:460] Webserver started at http://127.31.15.194:46707/ using document root <none> and password file <none>
I20260809 16:35:08.559015 11954 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:08.559064 11954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:08.559190 11954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:08.560261 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "2f15de40221a2f309ecc1497be9b2b0f"
server_key_iv: "5a5de6e9e7ba89b5f9c933027d03e6a0"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.560585 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "2f15de40221a2f309ecc1497be9b2b0f"
server_key_iv: "5a5de6e9e7ba89b5f9c933027d03e6a0"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.561836 11954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:08.562538 11981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.562703 11954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:08.562786 11954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "2f15de40221a2f309ecc1497be9b2b0f"
server_key_iv: "5a5de6e9e7ba89b5f9c933027d03e6a0"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.562847 11954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:08.579838 11954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:08.581594 11954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:08.581703 11954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:08.582146 11954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:08.582185 11954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.582216 11954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:08.582231 11954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.588186 11954 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:42705
I20260809 16:35:08.588243 12179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:42705 every 8 connection(s)
I20260809 16:35:08.588594 11954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:08.590479 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 11954
I20260809 16:35:08.590617 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:08.590835 31807 external_mini_cluster.cc:1428] Setting key 053ff46a0830051ab4e63ebd94b10125
I20260809 16:35:08.591907 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.582571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:32965 (local address 127.31.15.254:36519)
0809 16:35:08.582615 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.582617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.582833 (+ 216us) server_negotiation.cc:407] Connection header received
0809 16:35:08.583345 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.583346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.583359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.583407 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.584167 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.584934 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.586398 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.586706 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.586794 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.589212 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.589214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.589214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.589221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.590444 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.590893 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.590894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.590895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.590958 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.591226 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.591227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.591228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.591396 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.591460 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.591765 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.591837 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:35:08.592079 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.582567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.194:32965)
0809 16:35:08.582725 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.582758 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.583229 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.583511 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.583514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.583821 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.584083 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.584089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.585025 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.585026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.586288 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.586291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.586794 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.586795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.586902 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.587477 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.587487 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.589143 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.590523 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.590531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.590533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.590813 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.591018 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.591019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.591020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.591155 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.591458 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.591465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.591618 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.591728 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.591883 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 16:35:08.592512 12180 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36519
I20260809 16:35:08.592597 12180 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:08.592809 12180 heartbeater.cc:511] Master 127.31.15.254:36519 requested a full tablet report, sending...
I20260809 16:35:08.593189 11630 ts_manager.cc:194] Registered new tserver with Master: 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705)
I20260809 16:35:08.593667 11630 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:32965
I20260809 16:35:08.598914 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.593721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:50195 (local address 127.31.15.254:36519)
0809 16:35:08.593758 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.593759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.593881 (+ 122us) server_negotiation.cc:407] Connection header received
0809 16:35:08.593951 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.593952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.593963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.594012 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.594327 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.594769 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.595606 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.595781 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.595828 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.596682 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.596683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.596683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.596689 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.597912 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.598259 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.598259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.598260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.598302 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.598477 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.598478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.598478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.598600 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.598646 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.598786 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.598844 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:35:08.598969 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.593651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.194:50195)
0809 16:35:08.593825 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.593836 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.593892 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.594017 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.594017 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.594202 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.594266 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.594268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.594855 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.594856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.595534 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.595536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.595839 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.595840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.595896 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.596268 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.596273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.596601 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.597988 (+ 1387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.597989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.597989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.598192 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.598354 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.598355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.598355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.598417 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.598653 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.598654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.598700 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.598770 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.598863 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
I20260809 16:35:08.617681 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36519
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:33991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:08.698316 12190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:08.698525 12190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:08.698549 12190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:08.699666 12190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:08.699700 12190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:08.699718 12190 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 16:35:08.699736 12190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:08.701203 12190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:33991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.31.15.254:36519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:08.701388 12190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:08.701576 12190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:08.706321 12200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:08.706331 12198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.706524 12190 server_base.cc:1034] running on GCE node
W20260809 16:35:08.706332 12197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:08.706771 12190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:08.707013 12190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:08.708153 12190 hybrid_clock.cc:648] HybridClock initialized: now 1786293308708138 us; error 31 us; skew 500 ppm
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:08.710640 12190 init.cc:377] Logged in from keytab as oryx/127.31.15.195@KRBTEST.COM (short username oryx)
I20260809 16:35:08.711040 12190 webserver.cc:460] Webserver started at http://127.31.15.195:42921/ using document root <none> and password file <none>
I20260809 16:35:08.711216 12190 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:08.711254 12190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:08.711392 12190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:08.712397 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "379ddbf585384edcb71c6b3f4d97b84e"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "f77281f21a887b33e73c6ef07b4db15a"
server_key_iv: "2ad618d75784865be24087414b40e364"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.712700 12190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "379ddbf585384edcb71c6b3f4d97b84e"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "f77281f21a887b33e73c6ef07b4db15a"
server_key_iv: "2ad618d75784865be24087414b40e364"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.713917 12190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:08.714520 12214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.714675 12190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:08.714758 12190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "379ddbf585384edcb71c6b3f4d97b84e"
format_stamp: "Formatted at 2026-08-09 16:35:08 on dist-test-slave-21b9"
server_key: "f77281f21a887b33e73c6ef07b4db15a"
server_key_iv: "2ad618d75784865be24087414b40e364"
server_key_version: "encryptionkey@0"
I20260809 16:35:08.714813 12190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:08.722882 12190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:08.724638 12190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:08.724745 12190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:08.725257 12190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:08.725343 12190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.725385 12190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:08.725404 12190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.731705 12190 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:35389
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:08.731805 12397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:35389 every 8 connection(s)
I20260809 16:35:08.732080 12190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:08.732618 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 12190
I20260809 16:35:08.732708 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:08.732893 31807 external_mini_cluster.cc:1428] Setting key dd58abd830a25119cd1644da51679b70
I20260809 16:35:08.734824 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.725602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:45469 (local address 127.31.15.254:36519)
0809 16:35:08.725646 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.725648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.725862 (+ 214us) server_negotiation.cc:407] Connection header received
0809 16:35:08.726297 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.726298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.726312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.726383 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.727110 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.727896 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.729395 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.729713 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.729818 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.732237 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.732239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.732239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.732248 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.733835 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.734171 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.734171 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.734172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.734219 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.734378 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.734378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.734379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.734471 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.734516 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.734708 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.734763 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:35:08.734980 12226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.725603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.195:45469)
0809 16:35:08.725735 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.725768 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.726207 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.726445 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.726448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.726743 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.727010 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.727016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.728009 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.728010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.729310 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.729312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.729828 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.729829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.729967 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.730413 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.730422 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.732152 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.733909 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.733915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.733916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.734119 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.734259 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.734260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.734260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.734341 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.734508 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.734511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.734614 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.734689 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.734801 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260809 16:35:08.735479 12398 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36519
I20260809 16:35:08.735563 12398 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:08.735740 12398 heartbeater.cc:511] Master 127.31.15.254:36519 requested a full tablet report, sending...
I20260809 16:35:08.736141 11629 ts_manager.cc:194] Registered new tserver with Master: 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:08.736613 11629 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:45469
I20260809 16:35:08.740906 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.736480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:59249 (local address 127.31.15.254:36519)
0809 16:35:08.736514 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.736516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.736613 (+ 97us) server_negotiation.cc:407] Connection header received
0809 16:35:08.736706 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.736707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.736717 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.736771 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.737107 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.737548 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.738168 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.738330 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.738368 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.739101 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.739102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.739102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.739106 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.740135 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.740393 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.740393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.740394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.740435 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.740547 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.740547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.740548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.740626 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.740667 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.740775 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.740846 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:35:08.740908 12404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.736434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.31.15.195:59249)
0809 16:35:08.736564 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.736573 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.736624 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.736778 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.736779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.736979 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:08.737043 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.737045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.737625 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.737626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.738112 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.738113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.738375 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.738376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.738424 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.738778 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:08.738783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:08.739042 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:08.740194 (+ 1152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.740195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.740195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.740341 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.740472 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:08.740473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:08.740473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:08.740516 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:08.740658 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:08.740659 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:08.740690 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.740731 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.740833 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260809 16:35:08.744432 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:35:08.765503 11711 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:08.760562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.31.15.254:36519)
0809 16:35:08.760611 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.760613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.760622 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:08.760645 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.760646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.760655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.760711 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.761209 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.761693 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.762353 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.762541 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.762596 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.765419 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59354: BlockingRecv error: recv got EOF from 127.0.0.1:59354 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 16:35:08 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:08.772029 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.766245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.31.15.254:36519)
0809 16:35:08.766290 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.766292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.766344 (+ 52us) server_negotiation.cc:407] Connection header received
0809 16:35:08.766410 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.766411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.766422 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.766492 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:08.766873 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.767353 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.767967 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.768135 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.768181 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.770070 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:08.770071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:08.770071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:08.770077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:08.771127 (+ 1050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.771472 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.771473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.771473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.771518 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:08.771666 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:08.771666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:08.771666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:08.771752 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:08.771815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.771904 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.771957 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:35:08.773614 11629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:08.775147 11629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:08.781409 12431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:42705 (local address 127.0.0.1:58074)
0809 16:35:08.778012 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.778022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.778146 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.778444 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.778445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.778448 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:08.778507 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.778509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779638 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.779639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.780477 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.780479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781119 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.781207 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781221 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.781281 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.781334 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":113,"threads_started":1}
I20260809 16:35:08.781432 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.31.15.194:42705)
0809 16:35:08.777879 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.777883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.778087 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:35:08.778276 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.778279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.778295 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.778373 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:08.778593 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779565 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.780558 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781040 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781100 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781176 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.781304 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.781343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260809 16:35:08.781697 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43145 (local address 127.0.0.1:47332)
0809 16:35:08.777561 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.777598 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.777661 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.777918 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.777920 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.777922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:08.777999 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.778001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779153 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.779154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.780489 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.780492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781464 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.781530 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781534 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.781569 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.781609 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":84,"thread_start_us":45,"threads_started":1}
I20260809 16:35:08.781891 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.31.15.195:35389)
0809 16:35:08.777851 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.777856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.777865 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:08.777900 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.777902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.777915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.777994 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:08.778169 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779004 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.780599 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781312 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781386 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781470 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.781653 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.781697 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
I20260809 16:35:08.782088 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.31.15.193:43145)
0809 16:35:08.777720 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.777726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.777737 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:35:08.777780 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.777782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.777799 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.777918 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:08.778082 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779083 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.780600 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781312 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781823 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781916 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.781943 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.781985 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260809 16:35:08.782384 12055 tablet_service.cc:1467] Processing CreateTablet for tablet 42decd800d40477a97250b0d3c11cb3f (DEFAULT_TABLE table=test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.782742 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42decd800d40477a97250b0d3c11cb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.781702 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.777419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:35389 (local address 127.0.0.1:51766)
0809 16:35:08.777758 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.777766 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.777800 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.778010 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.778011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.778013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:08.778072 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.778074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.779081 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.779082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.780490 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.780492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.781503 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.781504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.781554 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.781557 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.781610 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.781641 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":24,"spinlock_wait_cycles":1280,"thread_start_us":119,"threads_started":1}
I20260809 16:35:08.783901 11845 tablet_service.cc:1467] Processing CreateTablet for tablet 42decd800d40477a97250b0d3c11cb3f (DEFAULT_TABLE table=test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.784037 12296 tablet_service.cc:1467] Processing CreateTablet for tablet 42decd800d40477a97250b0d3c11cb3f (DEFAULT_TABLE table=test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.784281 11845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42decd800d40477a97250b0d3c11cb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.784360 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42decd800d40477a97250b0d3c11cb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.786675 12440 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Bootstrap starting.
I20260809 16:35:08.787236 12443 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Bootstrap starting.
I20260809 16:35:08.787503 12440 tablet_bootstrap.cc:654] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.787791 12440 log.cc:824] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:08.787921 12443 tablet_bootstrap.cc:654] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.788268 12443 log.cc:824] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:08.788344 12445 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Bootstrap starting.
I20260809 16:35:08.788678 12440 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: No bootstrap required, opened a new log
I20260809 16:35:08.788754 12440 ts_tablet_manager.cc:1397] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:35:08.788995 12445 tablet_bootstrap.cc:654] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.789321 12445 log.cc:824] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:08.790081 12445 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: No bootstrap required, opened a new log
I20260809 16:35:08.790156 12445 ts_tablet_manager.cc:1397] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:08.790620 12440 raft_consensus.cc:348] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.790787 12440 raft_consensus.cc:374] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.790822 12440 raft_consensus.cc:729] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.790940 12440 consensus_queue.cc:260] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.791260 12180 heartbeater.cc:503] Master 127.31.15.254:36519 was elected leader, sending a full tablet report...
I20260809 16:35:08.791404 12440 ts_tablet_manager.cc:1428] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:35:08.791867 12443 tablet_bootstrap.cc:492] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: No bootstrap required, opened a new log
I20260809 16:35:08.791937 12443 ts_tablet_manager.cc:1397] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 16:35:08.792562 12445 raft_consensus.cc:348] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.792708 12445 raft_consensus.cc:374] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.792740 12445 raft_consensus.cc:729] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17985b0360fe40258cb28f8796082519, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.792846 12445 consensus_queue.cc:260] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.793093 12445 ts_tablet_manager.cc:1428] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:35:08.793138 11942 heartbeater.cc:503] Master 127.31.15.254:36519 was elected leader, sending a full tablet report...
I20260809 16:35:08.793736 12443 raft_consensus.cc:348] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.793871 12443 raft_consensus.cc:374] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.793905 12443 raft_consensus.cc:729] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379ddbf585384edcb71c6b3f4d97b84e, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.794000 12443 consensus_queue.cc:260] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.794227 12443 ts_tablet_manager.cc:1428] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:35:08.794277 12398 heartbeater.cc:503] Master 127.31.15.254:36519 was elected leader, sending a full tablet report...
I20260809 16:35:08.833760 12448 raft_consensus.cc:482] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:08.833892 12448 raft_consensus.cc:504] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.834285 12448 leader_election.cc:288] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:08.838032 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.834558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:33051 (local address 127.31.15.195:35389)
0809 16:35:08.834604 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.834606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.834615 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:08.834642 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.834643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.834652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.834707 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:08.834853 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.835660 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837117 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.837676 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837747 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.837756 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.837925 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.837961 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:35:08.838032 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.834568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:51255 (local address 127.31.15.193:43145)
0809 16:35:08.834608 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.834610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.834716 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:35:08.834750 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.834751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.834761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.834799 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:08.834919 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.835686 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837131 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.837676 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837747 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.837756 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.837924 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.837961 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:35:08.838057 12453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.834466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43145 (local address 127.31.15.194:51255)
0809 16:35:08.834652 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.834660 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.834710 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.834801 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.834802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.834803 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:08.834862 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.834863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.835745 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.835745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.837038 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.837041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.837749 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.837851 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.837855 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.837908 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.837945 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":81,"thread_start_us":116,"threads_started":1}
I20260809 16:35:08.838387 11877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42decd800d40477a97250b0d3c11cb3f" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17985b0360fe40258cb28f8796082519" is_pre_election: true
I20260809 16:35:08.838061 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.834465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:35389 (local address 127.31.15.194:33051)
0809 16:35:08.834537 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:08.834548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:08.834589 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:08.834718 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:08.834719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:08.834720 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:08.834790 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.834792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.835734 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.835735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.837038 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:08.837041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.837749 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.837749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:08.837855 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.837857 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:08.837908 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:35:08.837945 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":32}
I20260809 16:35:08.838526 11877 raft_consensus.cc:2454] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 0.
I20260809 16:35:08.838820 11982 leader_election.cc:302] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:08.838797 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42decd800d40477a97250b0d3c11cb3f" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e" is_pre_election: true
I20260809 16:35:08.838932 12327 raft_consensus.cc:2454] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 0.
I20260809 16:35:08.838939 12448 raft_consensus.cc:2781] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:08.838990 12448 raft_consensus.cc:482] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:08.839010 12448 raft_consensus.cc:3037] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:35:08.839202 12181 tablet.cc:2369] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:35:08.840014 12448 raft_consensus.cc:504] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.840135 12448 leader_election.cc:288] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 election: Requested vote from peers 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:08.840283 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42decd800d40477a97250b0d3c11cb3f" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e"
I20260809 16:35:08.840318 11877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42decd800d40477a97250b0d3c11cb3f" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17985b0360fe40258cb28f8796082519"
I20260809 16:35:08.840355 12327 raft_consensus.cc:3037] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:08.840381 11877 raft_consensus.cc:3037] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:08.841059 12327 raft_consensus.cc:2454] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 1.
I20260809 16:35:08.841089 11877 raft_consensus.cc:2454] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 1.
I20260809 16:35:08.841219 11984 leader_election.cc:302] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 379ddbf585384edcb71c6b3f4d97b84e, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:08.841318 12448 raft_consensus.cc:2781] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:08.841477 12448 raft_consensus.cc:686] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 LEADER]: Becoming Leader. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Running, Role: LEADER
I20260809 16:35:08.841567 12448 consensus_queue.cc:237] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.842376 11629 catalog_manager.cc:5495] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:08.852810 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:08.850842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.31.15.194:42705)
0809 16:35:08.850885 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:08.850887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:08.850896 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:08.850947 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:08.850947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:08.850962 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:08.851011 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:08.851114 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.851594 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.852272 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:08.852451 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:08.852497 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:08.852597 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:08.852685 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:08.852736 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:08.852749 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 16:35:08.852763 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:35:08.854357 12327 raft_consensus.cc:1261] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Refusing update from remote peer 53181ecbfd6d47beb36876d3e1dbcdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:08.854362 11877 raft_consensus.cc:1261] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Refusing update from remote peer 53181ecbfd6d47beb36876d3e1dbcdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:08.854655 12448 consensus_queue.cc:1035] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:08.854785 12454 consensus_queue.cc:1035] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:08.859342 11630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:08.859433 11630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:08.860080 11630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Sending DeleteTablet for 3 replicas of tablet 42decd800d40477a97250b0d3c11cb3f
I20260809 16:35:08.860381 11845 tablet_service.cc:1514] Processing DeleteTablet for tablet 42decd800d40477a97250b0d3c11cb3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:08 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:47332
I20260809 16:35:08.860381 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet 42decd800d40477a97250b0d3c11cb3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:08 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 16:35:08.860687 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet 42decd800d40477a97250b0d3c11cb3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:08 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:58074
I20260809 16:35:08.860754 12476 tablet_replica.cc:330] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: stopping tablet replica
I20260809 16:35:08.860847 12476 raft_consensus.cc:2229] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:08.860833 11630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:08.860980 11630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:08.860994 12476 raft_consensus.cc:2258] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:08.861133 12479 tablet_replica.cc:330] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: stopping tablet replica
I20260809 16:35:08.861282 12479 raft_consensus.cc:2229] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:08.861445 12479 raft_consensus.cc:2258] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:08.861616 12476 ts_tablet_manager.cc:1905] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:08.861910 12479 ts_tablet_manager.cc:1905] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:08.862195 12296 tablet_service.cc:1467] Processing CreateTablet for tablet f2d7fe574c4b480180c02bdb6b517774 (DEFAULT_TABLE table=test-table [id=405d6368d77f49b18f309c127a368607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.862296 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d7fe574c4b480180c02bdb6b517774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.863173 12443 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Bootstrap starting.
I20260809 16:35:08.862161 11845 tablet_service.cc:1467] Processing CreateTablet for tablet f2d7fe574c4b480180c02bdb6b517774 (DEFAULT_TABLE table=test-table [id=405d6368d77f49b18f309c127a368607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.863524 12055 tablet_service.cc:1467] Processing CreateTablet for tablet f2d7fe574c4b480180c02bdb6b517774 (DEFAULT_TABLE table=test-table [id=405d6368d77f49b18f309c127a368607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:08.863598 11845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d7fe574c4b480180c02bdb6b517774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.863597 12443 tablet_bootstrap.cc:654] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.863633 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2d7fe574c4b480180c02bdb6b517774. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:08.864825 12445 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Bootstrap starting.
I20260809 16:35:08.864949 12440 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Bootstrap starting.
I20260809 16:35:08.865226 12445 tablet_bootstrap.cc:654] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.865389 12440 tablet_bootstrap.cc:654] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:08.866192 12476 ts_tablet_manager.cc:1918] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:08.866226 12479 ts_tablet_manager.cc:1918] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:08.866274 12479 log.cc:1196] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/42decd800d40477a97250b0d3c11cb3f
I20260809 16:35:08.866328 12476 log.cc:1196] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/42decd800d40477a97250b0d3c11cb3f
I20260809 16:35:08.866514 12479 ts_tablet_manager.cc:1939] T 42decd800d40477a97250b0d3c11cb3f P 17985b0360fe40258cb28f8796082519: Deleting consensus metadata
I20260809 16:35:08.866604 12476 ts_tablet_manager.cc:1939] T 42decd800d40477a97250b0d3c11cb3f P 379ddbf585384edcb71c6b3f4d97b84e: Deleting consensus metadata
I20260809 16:35:08.866884 12481 tablet_replica.cc:330] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: stopping tablet replica
I20260809 16:35:08.866895 11605 catalog_manager.cc:4841] TS 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145): tablet 42decd800d40477a97250b0d3c11cb3f (table test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]) successfully deleted
I20260809 16:35:08.866976 12481 raft_consensus.cc:2229] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:08.866974 11608 catalog_manager.cc:4841] TS 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389): tablet 42decd800d40477a97250b0d3c11cb3f (table test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]) successfully deleted
I20260809 16:35:08.867087 12481 raft_consensus.cc:2258] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:08.867410 12440 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: No bootstrap required, opened a new log
I20260809 16:35:08.867461 12440 ts_tablet_manager.cc:1397] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:35:08.867492 12445 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: No bootstrap required, opened a new log
I20260809 16:35:08.867535 12445 ts_tablet_manager.cc:1397] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:35:08.867605 12440 raft_consensus.cc:348] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.867666 12440 raft_consensus.cc:374] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.867668 12445 raft_consensus.cc:348] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.867686 12440 raft_consensus.cc:729] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.867728 12445 raft_consensus.cc:374] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.867746 12445 raft_consensus.cc:729] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17985b0360fe40258cb28f8796082519, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.867780 12445 consensus_queue.cc:260] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.867766 12440 consensus_queue.cc:260] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.867856 12445 ts_tablet_manager.cc:1428] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:08.867863 12440 ts_tablet_manager.cc:1428] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:08.867681 12481 ts_tablet_manager.cc:1905] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:08.869175 12481 ts_tablet_manager.cc:1918] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:08.869236 12481 log.cc:1196] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/42decd800d40477a97250b0d3c11cb3f
I20260809 16:35:08.869460 12481 ts_tablet_manager.cc:1939] T 42decd800d40477a97250b0d3c11cb3f P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting consensus metadata
I20260809 16:35:08.869784 11609 catalog_manager.cc:4841] TS 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705): tablet 42decd800d40477a97250b0d3c11cb3f (table test-table [id=4f7d2a5dd2ee4f30acb87db71cf08899]) successfully deleted
I20260809 16:35:08.871592 12443 tablet_bootstrap.cc:492] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: No bootstrap required, opened a new log
I20260809 16:35:08.871634 12443 ts_tablet_manager.cc:1397] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260809 16:35:08.871753 12443 raft_consensus.cc:348] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.871805 12443 raft_consensus.cc:374] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:08.871819 12443 raft_consensus.cc:729] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379ddbf585384edcb71c6b3f4d97b84e, State: Initialized, Role: FOLLOWER
I20260809 16:35:08.871850 12443 consensus_queue.cc:260] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:08.871941 12443 ts_tablet_manager.cc:1428] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260809 16:35:08.938817 11943 tablet.cc:2369] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:35:08.982692 12399 tablet.cc:2369] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:35:09.067667 12449 raft_consensus.cc:482] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:09.067777 12449 raft_consensus.cc:504] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.068130 12449 leader_election.cc:288] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.071041 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.068383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:56863 (local address 127.31.15.194:42705)
0809 16:35:09.068521 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.068523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.068535 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:35:09.068636 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.068637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.068649 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.068680 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:09.068911 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.069432 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070387 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.070746 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070808 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.070817 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.070939 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.070975 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 16:35:09.071108 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.068311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:42705 (local address 127.31.15.193:56863)
0809 16:35:09.068462 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.068476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:09.068544 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:09.068756 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:09.068757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:09.068759 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:09.068831 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.068833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.069510 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.069511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.070306 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.070308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.070816 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.070881 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.070885 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:09.070929 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:35:09.070969 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260809 16:35:09.071108 12437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.068444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:54831 (local address 127.31.15.195:35389)
0809 16:35:09.068577 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.068579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.068611 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:35:09.068636 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.068637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.068647 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.068697 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:09.068856 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.069359 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070420 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.070769 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070838 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.070847 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.070996 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.071034 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 16:35:09.071133 12491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.068382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:35389 (local address 127.31.15.193:54831)
0809 16:35:09.068499 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.068520 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:09.068575 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:09.068717 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:09.068718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:09.068720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:09.068793 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.068795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.069467 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.069468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.070322 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.070325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.070844 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.070845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.070923 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.070926 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:09.070972 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:35:09.071017 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":44,"thread_start_us":68,"threads_started":1}
I20260809 16:35:09.071434 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d7fe574c4b480180c02bdb6b517774" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e" is_pre_election: true
I20260809 16:35:09.071542 12327 raft_consensus.cc:2454] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.071635 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d7fe574c4b480180c02bdb6b517774" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" is_pre_election: true
I20260809 16:35:09.071723 11772 leader_election.cc:302] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 379ddbf585384edcb71c6b3f4d97b84e; no voters:
I20260809 16:35:09.071789 12099 raft_consensus.cc:2454] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.071825 12449 raft_consensus.cc:2781] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:09.071861 12449 raft_consensus.cc:482] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:09.071883 12449 raft_consensus.cc:3037] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.072554 12449 raft_consensus.cc:504] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.072675 12449 leader_election.cc:288] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Requested vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.072831 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d7fe574c4b480180c02bdb6b517774" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
I20260809 16:35:09.072875 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2d7fe574c4b480180c02bdb6b517774" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e"
I20260809 16:35:09.072912 12099 raft_consensus.cc:3037] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.072937 12327 raft_consensus.cc:3037] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.073482 12327 raft_consensus.cc:2454] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.073638 12099 raft_consensus.cc:2454] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.073644 11772 leader_election.cc:302] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 379ddbf585384edcb71c6b3f4d97b84e; no voters:
I20260809 16:35:09.073745 12449 raft_consensus.cc:2781] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:09.073868 12449 raft_consensus.cc:686] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 1 LEADER]: Becoming Leader. State: Replica: 17985b0360fe40258cb28f8796082519, State: Running, Role: LEADER
I20260809 16:35:09.073936 12449 consensus_queue.cc:237] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.074690 11629 catalog_manager.cc:5495] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17985b0360fe40258cb28f8796082519 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } health_report { overall_health: UNKNOWN } } }
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293308, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.254@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:09.122474 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.116018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59374 (local address 127.31.15.254:36519)
0809 16:35:09.116100 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.116103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.116199 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:35:09.116253 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.116253 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.116262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.116314 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:09.116780 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.117269 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.117992 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.118173 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.118224 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.120266 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:09.120268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:09.120268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:09.120274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:09.121515 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:09.121865 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:09.121866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:09.121866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:09.121919 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:09.122069 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:09.122069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:09.122070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:09.122158 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:09.122200 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.122305 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.122368 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 16:35:09.122545 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.115978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36519 (local address 127.0.0.1:59374)
0809 16:35:09.116126 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.116137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:09.116206 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:09.116326 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:09.116327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:09.116638 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:09.116714 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.116716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.117338 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.117339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.117925 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.117927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.118227 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.118228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.118265 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.118763 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:09.118767 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:09.120183 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:09.121597 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:09.121604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:09.121606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:09.121804 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:09.121972 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:09.121973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:09.121974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:09.122032 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:09.122197 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:09.122203 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:09.122243 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 16:35:09.122289 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:35:09.122428 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":52,"thread_start_us":82,"threads_started":1}
I20260809 16:35:09.124431 11629 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:59374:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 16:35:09.124643 11629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:09.126063 11845 tablet_service.cc:1467] Processing CreateTablet for tablet bd82ad9ea5934cb5be4fb8d2206a71e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84b17ffc5424f2da3b324d29b2156b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:35:09.126097 12055 tablet_service.cc:1467] Processing CreateTablet for tablet bd82ad9ea5934cb5be4fb8d2206a71e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84b17ffc5424f2da3b324d29b2156b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:35:09.126255 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd82ad9ea5934cb5be4fb8d2206a71e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.126255 11845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd82ad9ea5934cb5be4fb8d2206a71e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.127399 12440 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: Bootstrap starting.
I20260809 16:35:09.127429 12445 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: Bootstrap starting.
I20260809 16:35:09.127732 12440 tablet_bootstrap.cc:654] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.127768 12445 tablet_bootstrap.cc:654] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.128172 12296 tablet_service.cc:1467] Processing CreateTablet for tablet bd82ad9ea5934cb5be4fb8d2206a71e6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b84b17ffc5424f2da3b324d29b2156b8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 16:35:09.128245 12440 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: No bootstrap required, opened a new log
I20260809 16:35:09.128288 12440 ts_tablet_manager.cc:1397] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:09.128315 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd82ad9ea5934cb5be4fb8d2206a71e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.128403 12440 raft_consensus.cc:348] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.128469 12440 raft_consensus.cc:374] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.128485 12440 raft_consensus.cc:729] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.128520 12440 consensus_queue.cc:260] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.128615 12440 ts_tablet_manager.cc:1428] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:09.128641 12470 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.128731 12470 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.129631 12443 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: Bootstrap starting.
I20260809 16:35:09.129849 12445 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: No bootstrap required, opened a new log
I20260809 16:35:09.129900 12445 ts_tablet_manager.cc:1397] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:09.130076 12443 tablet_bootstrap.cc:654] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.130054 12445 raft_consensus.cc:348] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.130116 12445 raft_consensus.cc:374] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.130136 12445 raft_consensus.cc:729] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17985b0360fe40258cb28f8796082519, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.130190 12445 consensus_queue.cc:260] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.130283 12445 ts_tablet_manager.cc:1428] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:09.130326 12449 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.130451 12449 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.130772 12443 tablet_bootstrap.cc:492] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: No bootstrap required, opened a new log
I20260809 16:35:09.130825 12443 ts_tablet_manager.cc:1397] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:09.130975 12443 raft_consensus.cc:348] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.131045 12443 raft_consensus.cc:374] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.131062 12443 raft_consensus.cc:729] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379ddbf585384edcb71c6b3f4d97b84e, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.131104 12443 consensus_queue.cc:260] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.131189 12443 ts_tablet_manager.cc:1428] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:09.131253 12450 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.131378 12450 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.180456 12449 raft_consensus.cc:482] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:09.180557 12449 raft_consensus.cc:504] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.180698 12449 leader_election.cc:288] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.180905 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd82ad9ea5934cb5be4fb8d2206a71e6" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" is_pre_election: true
I20260809 16:35:09.180917 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd82ad9ea5934cb5be4fb8d2206a71e6" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e" is_pre_election: true
I20260809 16:35:09.181002 12099 raft_consensus.cc:2454] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.181016 12327 raft_consensus.cc:2454] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.181157 11772 leader_election.cc:302] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 379ddbf585384edcb71c6b3f4d97b84e; no voters:
I20260809 16:35:09.181262 12449 raft_consensus.cc:2781] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:09.181301 12449 raft_consensus.cc:482] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:09.181324 12449 raft_consensus.cc:3037] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.181996 12449 raft_consensus.cc:504] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.182111 12449 leader_election.cc:288] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Requested vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.182253 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd82ad9ea5934cb5be4fb8d2206a71e6" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
I20260809 16:35:09.182332 12099 raft_consensus.cc:3037] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.182297 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd82ad9ea5934cb5be4fb8d2206a71e6" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e"
I20260809 16:35:09.182377 12327 raft_consensus.cc:3037] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.182907 12099 raft_consensus.cc:2454] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.182971 12327 raft_consensus.cc:2454] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.183100 11773 leader_election.cc:302] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:09.183199 12449 raft_consensus.cc:2781] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:09.183244 12449 raft_consensus.cc:686] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [term 1 LEADER]: Becoming Leader. State: Replica: 17985b0360fe40258cb28f8796082519, State: Running, Role: LEADER
I20260809 16:35:09.183310 12449 consensus_queue.cc:237] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.183604 12492 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: TxnStatusTablet state changed. Reason: New leader 17985b0360fe40258cb28f8796082519. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.183681 12492 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:35:09.183894 11629 catalog_manager.cc:5495] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17985b0360fe40258cb28f8796082519 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:09.184185 12511 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 16:35:09.184299 12511 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 16:35:09.204180 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.201539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47346 (local address 127.31.15.193:43145)
0809 16:35:09.201578 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.201580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.201607 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:35:09.201693 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.201694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.201710 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.201766 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:09.201899 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.202370 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.203472 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.203654 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.203706 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.203891 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.203992 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.204039 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.204107 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.204122 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:35:09.204234 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.201476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43145 (local address 127.0.0.1:47346)
0809 16:35:09.201564 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.201575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:09.201625 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:09.201785 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:09.201786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:09.201787 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:35:09.201843 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.201845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.202435 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.202436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.203399 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.203402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.203719 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.203719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.203786 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.203806 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.204049 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.204054 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:35:09.204102 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:35:09.204149 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:35:09.205276 12327 raft_consensus.cc:1261] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.205546 12099 raft_consensus.cc:1261] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.205549 12449 consensus_queue.cc:1035] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.205776 12449 consensus_queue.cc:1035] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.206272 12450 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: TxnStatusTablet state changed. Reason: New leader 17985b0360fe40258cb28f8796082519. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.206363 12450 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.206847 12449 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.206926 12449 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:35:09.206928 12470 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: TxnStatusTablet state changed. Reason: New leader 17985b0360fe40258cb28f8796082519. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.207012 12470 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.207217 12463 mvcc.cc:205] Tried to move back new op lower bound from 7316657394502832128 to 7316657394415190016. Current Snapshot: MvccSnapshot[applied={T|T < 7316657394502832128 or (T in {7316657394502832128})}]
I20260809 16:35:09.207970 12450 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.208060 12450 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 379ddbf585384edcb71c6b3f4d97b84e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.208781 12492 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.208873 12492 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 17985b0360fe40258cb28f8796082519: This TxnStatusTablet replica's current role is: LEADER
I20260809 16:35:09.209172 12462 mvcc.cc:205] Tried to move back new op lower bound from 7316657394502832128 to 7316657394415190016. Current Snapshot: MvccSnapshot[applied={T|T < 7316657394502832128}]
I20260809 16:35:09.209259 12470 tablet_replica.cc:439] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } } }
I20260809 16:35:09.209316 12470 tablet_replica.cc:442] T bd82ad9ea5934cb5be4fb8d2206a71e6 P 53181ecbfd6d47beb36876d3e1dbcdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 16:35:09.211126 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.208699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.31.15.193:43145)
0809 16:35:09.208732 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.208734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.208740 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:09.208759 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.208760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.208768 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.208816 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:09.208968 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.209693 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.210443 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.210735 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.210802 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.210927 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.210959 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.211010 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.211047 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.211063 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:35:09.216029 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.213550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:32959 (local address 127.31.15.193:43145)
0809 16:35:09.213588 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.213590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.213696 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:35:09.213752 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.213752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.213760 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.213801 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:09.213935 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.214347 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.215417 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.215598 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.215649 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.215815 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.215837 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.215867 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.215932 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.215946 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:35:09.216135 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.213469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43145 (local address 127.31.15.193:32959)
0809 16:35:09.213632 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.213642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:09.213701 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:09.213807 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:09.213807 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:09.213809 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 16:35:09.213881 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.213882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.214412 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.214413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.215347 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:09.215349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.215663 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.215664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:09.215726 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.215742 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.215882 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.215884 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:09.215930 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:35:09.215970 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260809 16:35:09.217943 12099 raft_consensus.cc:1261] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.217971 12327 raft_consensus.cc:1261] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.218190 12492 consensus_queue.cc:1035] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.218286 12449 consensus_queue.cc:1035] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.222368 12465 mvcc.cc:205] Tried to move back new op lower bound from 7316657394555301888 to 7316657393967529984. Current Snapshot: MvccSnapshot[applied={T|T < 7316657394555301888}]
I20260809 16:35:09.232151 11630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.232251 11630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.232868 11630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Sending DeleteTablet for 3 replicas of tablet f2d7fe574c4b480180c02bdb6b517774
I20260809 16:35:09.233108 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d7fe574c4b480180c02bdb6b517774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 16:35:09.233131 11845 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d7fe574c4b480180c02bdb6b517774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:47332
I20260809 16:35:09.233206 12479 tablet_replica.cc:330] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: stopping tablet replica
I20260809 16:35:09.233217 12476 tablet_replica.cc:330] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: stopping tablet replica
I20260809 16:35:09.233258 12479 raft_consensus.cc:2229] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:09.233240 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet f2d7fe574c4b480180c02bdb6b517774 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:58074
I20260809 16:35:09.233283 12476 raft_consensus.cc:2229] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.233316 12481 tablet_replica.cc:330] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: stopping tablet replica
I20260809 16:35:09.233335 12476 raft_consensus.cc:2258] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.233341 12479 raft_consensus.cc:2258] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.233364 12481 raft_consensus.cc:2229] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.233400 12481 raft_consensus.cc:2258] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.233672 12479 ts_tablet_manager.cc:1905] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.235105 12481 ts_tablet_manager.cc:1905] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.235353 12479 ts_tablet_manager.cc:1918] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:35:09.235402 12479 log.cc:1196] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/f2d7fe574c4b480180c02bdb6b517774
I20260809 16:35:09.235599 12479 ts_tablet_manager.cc:1939] T f2d7fe574c4b480180c02bdb6b517774 P 17985b0360fe40258cb28f8796082519: Deleting consensus metadata
I20260809 16:35:09.235867 11605 catalog_manager.cc:4841] TS 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145): tablet f2d7fe574c4b480180c02bdb6b517774 (table test-table [id=405d6368d77f49b18f309c127a368607]) successfully deleted
I20260809 16:35:09.236544 12481 ts_tablet_manager.cc:1918] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:35:09.236584 12481 log.cc:1196] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/f2d7fe574c4b480180c02bdb6b517774
I20260809 16:35:09.236711 12481 ts_tablet_manager.cc:1939] T f2d7fe574c4b480180c02bdb6b517774 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting consensus metadata
I20260809 16:35:09.236996 12476 ts_tablet_manager.cc:1905] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.237352 11609 catalog_manager.cc:4841] TS 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705): tablet f2d7fe574c4b480180c02bdb6b517774 (table test-table [id=405d6368d77f49b18f309c127a368607]) successfully deleted
I20260809 16:35:09.238161 12476 ts_tablet_manager.cc:1918] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:35:09.238214 12476 log.cc:1196] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/f2d7fe574c4b480180c02bdb6b517774
I20260809 16:35:09.238381 12476 ts_tablet_manager.cc:1939] T f2d7fe574c4b480180c02bdb6b517774 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting consensus metadata
I20260809 16:35:09.238670 11608 catalog_manager.cc:4841] TS 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389): tablet f2d7fe574c4b480180c02bdb6b517774 (table test-table [id=405d6368d77f49b18f309c127a368607]) successfully deleted
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[11561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 16:35:09.269258 11711 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:09.263027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59382 (local address 127.31.15.254:36519)
0809 16:35:09.263097 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.263099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.263226 (+ 127us) server_negotiation.cc:407] Connection header received
0809 16:35:09.263329 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.263331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.263344 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.263421 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:09.264141 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.264799 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.265884 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.266093 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.266145 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.269186 (+ 3041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59382: BlockingRecv error: recv got EOF from 127.0.0.1:59382 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[11561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM
I20260809 16:35:09.276901 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.270072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59386 (local address 127.31.15.254:36519)
0809 16:35:09.270109 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.270110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.270137 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:35:09.270184 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.270185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.270196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.270262 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:09.270680 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.271488 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.272121 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.272409 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.272479 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.274392 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:09.274393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:09.274393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:09.274401 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:09.275872 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:09.276196 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:09.276197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:09.276197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:09.276262 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:09.276418 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:09.276419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:09.276419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:09.276554 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:09.276655 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.276730 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.276807 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:35:09.278399 11630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:09.278580 11630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:09.279973 11845 tablet_service.cc:1467] Processing CreateTablet for tablet c50467eeb4984d1988cb0191a4b40456 (DEFAULT_TABLE table=test-table [id=fb7c455b78ee49cd8985808019692243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.280081 12055 tablet_service.cc:1467] Processing CreateTablet for tablet c50467eeb4984d1988cb0191a4b40456 (DEFAULT_TABLE table=test-table [id=fb7c455b78ee49cd8985808019692243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.280128 11845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50467eeb4984d1988cb0191a4b40456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.280190 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50467eeb4984d1988cb0191a4b40456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.281448 12440 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Bootstrap starting.
I20260809 16:35:09.281469 12296 tablet_service.cc:1467] Processing CreateTablet for tablet c50467eeb4984d1988cb0191a4b40456 (DEFAULT_TABLE table=test-table [id=fb7c455b78ee49cd8985808019692243]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.281574 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c50467eeb4984d1988cb0191a4b40456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.281806 12440 tablet_bootstrap.cc:654] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.282090 12445 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Bootstrap starting.
I20260809 16:35:09.282490 12445 tablet_bootstrap.cc:654] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.282675 12443 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Bootstrap starting.
I20260809 16:35:09.283044 12443 tablet_bootstrap.cc:654] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.283607 12440 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: No bootstrap required, opened a new log
I20260809 16:35:09.283633 12443 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: No bootstrap required, opened a new log
I20260809 16:35:09.283666 12440 ts_tablet_manager.cc:1397] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:09.283670 12443 ts_tablet_manager.cc:1397] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:09.283805 12443 raft_consensus.cc:348] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.283811 12440 raft_consensus.cc:348] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.283881 12440 raft_consensus.cc:374] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.283881 12443 raft_consensus.cc:374] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.283913 12443 raft_consensus.cc:729] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379ddbf585384edcb71c6b3f4d97b84e, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.283913 12440 raft_consensus.cc:729] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.283965 12443 consensus_queue.cc:260] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.283965 12440 consensus_queue.cc:260] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.284055 12443 ts_tablet_manager.cc:1428] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:09.284068 12440 ts_tablet_manager.cc:1428] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:09.284852 12445 tablet_bootstrap.cc:492] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: No bootstrap required, opened a new log
I20260809 16:35:09.284909 12445 ts_tablet_manager.cc:1397] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 16:35:09.285079 12445 raft_consensus.cc:348] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.285171 12445 raft_consensus.cc:374] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.285198 12445 raft_consensus.cc:729] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17985b0360fe40258cb28f8796082519, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.285250 12445 consensus_queue.cc:260] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.285363 12445 ts_tablet_manager.cc:1428] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:09.511168 12470 raft_consensus.cc:482] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:09.511302 12470 raft_consensus.cc:504] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.511461 12470 leader_election.cc:288] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.511696 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50467eeb4984d1988cb0191a4b40456" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e" is_pre_election: true
I20260809 16:35:09.511696 11877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50467eeb4984d1988cb0191a4b40456" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17985b0360fe40258cb28f8796082519" is_pre_election: true
I20260809 16:35:09.511811 12327 raft_consensus.cc:2454] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 0.
I20260809 16:35:09.511812 11877 raft_consensus.cc:2454] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 0.
I20260809 16:35:09.511958 11982 leader_election.cc:302] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:09.512061 12470 raft_consensus.cc:2781] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:09.512106 12470 raft_consensus.cc:482] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:09.512125 12470 raft_consensus.cc:3037] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.512843 12470 raft_consensus.cc:504] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.512967 12470 leader_election.cc:288] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 election: Requested vote from peers 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.513121 11877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50467eeb4984d1988cb0191a4b40456" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17985b0360fe40258cb28f8796082519"
I20260809 16:35:09.513134 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c50467eeb4984d1988cb0191a4b40456" candidate_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e"
I20260809 16:35:09.513197 11877 raft_consensus.cc:3037] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.513206 12327 raft_consensus.cc:3037] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.513810 12327 raft_consensus.cc:2454] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 1.
I20260809 16:35:09.513808 11877 raft_consensus.cc:2454] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53181ecbfd6d47beb36876d3e1dbcdc5 in term 1.
I20260809 16:35:09.513983 11982 leader_election.cc:302] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 379ddbf585384edcb71c6b3f4d97b84e, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:09.514110 12470 raft_consensus.cc:2781] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:09.514225 12470 raft_consensus.cc:686] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 LEADER]: Becoming Leader. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Running, Role: LEADER
I20260809 16:35:09.514281 12470 consensus_queue.cc:237] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.514786 11629 catalog_manager.cc:5495] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:09.568192 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.566233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.31.15.194:42705)
0809 16:35:09.566288 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.566290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.566297 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:09.566319 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.566320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.566329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.566378 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:09.566487 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.566966 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.567709 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.567885 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.567928 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.568016 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.568049 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.568080 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.568116 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.568132 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:35:09.568939 11877 raft_consensus.cc:1261] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Refusing update from remote peer 53181ecbfd6d47beb36876d3e1dbcdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.568990 12327 raft_consensus.cc:1261] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Refusing update from remote peer 53181ecbfd6d47beb36876d3e1dbcdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.569212 12554 consensus_queue.cc:1035] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.569336 12470 consensus_queue.cc:1035] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.571889 11629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.571969 11629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.572512 11629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Sending DeleteTablet for 3 replicas of tablet c50467eeb4984d1988cb0191a4b40456
I20260809 16:35:09.572733 11845 tablet_service.cc:1514] Processing DeleteTablet for tablet c50467eeb4984d1988cb0191a4b40456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:47332
I20260809 16:35:09.572788 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet c50467eeb4984d1988cb0191a4b40456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:58074
I20260809 16:35:09.572888 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet c50467eeb4984d1988cb0191a4b40456 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 16:35:09.572973 12476 tablet_replica.cc:330] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: stopping tablet replica
I20260809 16:35:09.572980 12479 tablet_replica.cc:330] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: stopping tablet replica
I20260809 16:35:09.573047 12479 raft_consensus.cc:2229] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.573050 12476 raft_consensus.cc:2229] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.573093 12476 raft_consensus.cc:2258] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.573093 12479 raft_consensus.cc:2258] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.573164 12481 tablet_replica.cc:330] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: stopping tablet replica
I20260809 16:35:09.573220 12481 raft_consensus.cc:2229] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:09.573273 12481 raft_consensus.cc:2258] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.573349 12476 ts_tablet_manager.cc:1905] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.573534 12479 ts_tablet_manager.cc:1905] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.573717 11629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 16:35:09.573832 12481 ts_tablet_manager.cc:1905] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:35:09.573849 11629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:09.574764 12476 ts_tablet_manager.cc:1918] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:09.574815 12476 log.cc:1196] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/c50467eeb4984d1988cb0191a4b40456
I20260809 16:35:09.574966 12479 ts_tablet_manager.cc:1918] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:09.575016 12479 log.cc:1196] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/c50467eeb4984d1988cb0191a4b40456
I20260809 16:35:09.575021 12476 ts_tablet_manager.cc:1939] T c50467eeb4984d1988cb0191a4b40456 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting consensus metadata
I20260809 16:35:09.575223 12479 ts_tablet_manager.cc:1939] T c50467eeb4984d1988cb0191a4b40456 P 17985b0360fe40258cb28f8796082519: Deleting consensus metadata
I20260809 16:35:09.575251 12481 ts_tablet_manager.cc:1918] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:09.575330 12481 log.cc:1196] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/c50467eeb4984d1988cb0191a4b40456
I20260809 16:35:09.575506 12481 ts_tablet_manager.cc:1939] T c50467eeb4984d1988cb0191a4b40456 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting consensus metadata
I20260809 16:35:09.575515 11605 catalog_manager.cc:4841] TS 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145): tablet c50467eeb4984d1988cb0191a4b40456 (table test-table [id=fb7c455b78ee49cd8985808019692243]) successfully deleted
I20260809 16:35:09.575584 11845 tablet_service.cc:1467] Processing CreateTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 (DEFAULT_TABLE table=test-table [id=7846188897004a2282def2302f842c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.575686 11845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153b2946e0b54ee491a4659c5cf2d2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.576478 12445 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: Bootstrap starting.
I20260809 16:35:09.576532 11608 catalog_manager.cc:4841] TS 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389): tablet c50467eeb4984d1988cb0191a4b40456 (table test-table [id=fb7c455b78ee49cd8985808019692243]) successfully deleted
I20260809 16:35:09.576601 12296 tablet_service.cc:1467] Processing CreateTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 (DEFAULT_TABLE table=test-table [id=7846188897004a2282def2302f842c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.576649 11609 catalog_manager.cc:4841] TS 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705): tablet c50467eeb4984d1988cb0191a4b40456 (table test-table [id=fb7c455b78ee49cd8985808019692243]) successfully deleted
I20260809 16:35:09.576697 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153b2946e0b54ee491a4659c5cf2d2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.576805 12445 tablet_bootstrap.cc:654] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.576793 12055 tablet_service.cc:1467] Processing CreateTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 (DEFAULT_TABLE table=test-table [id=7846188897004a2282def2302f842c1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:09.576876 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 153b2946e0b54ee491a4659c5cf2d2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:09.577332 12445 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: No bootstrap required, opened a new log
I20260809 16:35:09.577378 12445 ts_tablet_manager.cc:1397] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:09.577483 12445 raft_consensus.cc:348] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.577548 12445 raft_consensus.cc:374] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.577565 12445 raft_consensus.cc:729] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17985b0360fe40258cb28f8796082519, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.577610 12445 consensus_queue.cc:260] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.577699 12445 ts_tablet_manager.cc:1428] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:09.577811 12443 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Bootstrap starting.
I20260809 16:35:09.577877 12440 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Bootstrap starting.
I20260809 16:35:09.578212 12440 tablet_bootstrap.cc:654] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.578480 12443 tablet_bootstrap.cc:654] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:09.578758 12440 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: No bootstrap required, opened a new log
I20260809 16:35:09.578805 12440 ts_tablet_manager.cc:1397] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:09.578902 12440 raft_consensus.cc:348] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.578965 12440 raft_consensus.cc:374] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.578984 12440 raft_consensus.cc:729] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53181ecbfd6d47beb36876d3e1dbcdc5, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.579028 12440 consensus_queue.cc:260] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.579066 12443 tablet_bootstrap.cc:492] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: No bootstrap required, opened a new log
I20260809 16:35:09.579099 12443 ts_tablet_manager.cc:1397] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:09.579123 12440 ts_tablet_manager.cc:1428] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:09.579229 12443 raft_consensus.cc:348] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.579321 12443 raft_consensus.cc:374] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:09.579344 12443 raft_consensus.cc:729] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379ddbf585384edcb71c6b3f4d97b84e, State: Initialized, Role: FOLLOWER
I20260809 16:35:09.579387 12443 consensus_queue.cc:260] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.579484 12443 ts_tablet_manager.cc:1428] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:09.652149 12526 raft_consensus.cc:482] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:09.652251 12526 raft_consensus.cc:504] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.652401 12526 leader_election.cc:288] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.652653 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153b2946e0b54ee491a4659c5cf2d2f2" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" is_pre_election: true
I20260809 16:35:09.652666 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153b2946e0b54ee491a4659c5cf2d2f2" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e" is_pre_election: true
I20260809 16:35:09.652755 12099 raft_consensus.cc:2454] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.652757 12327 raft_consensus.cc:2454] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 0.
I20260809 16:35:09.652915 11773 leader_election.cc:302] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:09.653039 12526 raft_consensus.cc:2781] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:09.653079 12526 raft_consensus.cc:482] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:09.653093 12526 raft_consensus.cc:3037] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.653825 12526 raft_consensus.cc:504] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.653944 12526 leader_election.cc:288] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Requested vote from peers 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705), 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389)
I20260809 16:35:09.654115 12099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153b2946e0b54ee491a4659c5cf2d2f2" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5"
I20260809 16:35:09.654143 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "153b2946e0b54ee491a4659c5cf2d2f2" candidate_uuid: "17985b0360fe40258cb28f8796082519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379ddbf585384edcb71c6b3f4d97b84e"
I20260809 16:35:09.654194 12099 raft_consensus.cc:3037] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.654214 12327 raft_consensus.cc:3037] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:09.654803 12099 raft_consensus.cc:2454] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.654857 12327 raft_consensus.cc:2454] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17985b0360fe40258cb28f8796082519 in term 1.
I20260809 16:35:09.654979 11773 leader_election.cc:302] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17985b0360fe40258cb28f8796082519, 53181ecbfd6d47beb36876d3e1dbcdc5; no voters:
I20260809 16:35:09.655074 12526 raft_consensus.cc:2781] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:09.655124 12526 raft_consensus.cc:686] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 1 LEADER]: Becoming Leader. State: Replica: 17985b0360fe40258cb28f8796082519, State: Running, Role: LEADER
I20260809 16:35:09.655164 12526 consensus_queue.cc:237] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } }
I20260809 16:35:09.655758 11630 catalog_manager.cc:5495] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 17985b0360fe40258cb28f8796082519 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "17985b0360fe40258cb28f8796082519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17985b0360fe40258cb28f8796082519" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 } health_report { overall_health: UNKNOWN } } }
I20260809 16:35:09.673645 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.671752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.31.15.193:43145)
0809 16:35:09.671797 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:09.671799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:09.671808 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:09.671848 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:09.671848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:09.671856 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:09.671892 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:09.671978 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.672436 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.673144 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:09.673329 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:09.673390 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:09.673467 (+ 77us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:09.673493 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:09.673522 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:09.673562 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:35:09.673576 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:35:09.675678 12099 raft_consensus.cc:1261] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.675695 12327 raft_consensus.cc:1261] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Refusing update from remote peer 17985b0360fe40258cb28f8796082519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:09.675863 12449 consensus_queue.cc:1035] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53181ecbfd6d47beb36876d3e1dbcdc5" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.675976 12526 consensus_queue.cc:1035] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379ddbf585384edcb71c6b3f4d97b84e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 35389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:09.684913 11629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.684993 11629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:09.685501 11629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82ff5bdc6ff4449c946026e32daf2475: Sending DeleteTablet for 3 replicas of tablet 153b2946e0b54ee491a4659c5cf2d2f2
I20260809 16:35:09.685709 11843 tablet_service.cc:1514] Processing DeleteTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:47332
I20260809 16:35:09.685794 12479 tablet_replica.cc:330] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519: stopping tablet replica
I20260809 16:35:09.685796 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 16:35:09.685846 12479 raft_consensus.cc:2229] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:09.685880 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet 153b2946e0b54ee491a4659c5cf2d2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:09 UTC) from {username='oryx', principal='oryx/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:58074
I20260809 16:35:09.685904 12479 raft_consensus.cc:2258] T 153b2946e0b54ee491a4659c5cf2d2f2 P 17985b0360fe40258cb28f8796082519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.685917 12476 tablet_replica.cc:330] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: stopping tablet replica
I20260809 16:35:09.685966 12481 tablet_replica.cc:330] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: stopping tablet replica
I20260809 16:35:09.685976 12476 raft_consensus.cc:2229] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.686007 12476 raft_consensus.cc:2258] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.686012 12481 raft_consensus.cc:2229] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:09.686036 12481 raft_consensus.cc:2258] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:09.686096 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 11747
I20260809 16:35:09.686784 12481 ts_tablet_manager.cc:1905] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.688805 12481 ts_tablet_manager.cc:1918] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:35:09.688865 12481 log.cc:1196] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/153b2946e0b54ee491a4659c5cf2d2f2
I20260809 16:35:09.689062 12481 ts_tablet_manager.cc:1939] T 153b2946e0b54ee491a4659c5cf2d2f2 P 53181ecbfd6d47beb36876d3e1dbcdc5: Deleting consensus metadata
I20260809 16:35:09.689066 12476 ts_tablet_manager.cc:1905] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:09.689344 11609 catalog_manager.cc:4841] TS 53181ecbfd6d47beb36876d3e1dbcdc5 (127.31.15.194:42705): tablet 153b2946e0b54ee491a4659c5cf2d2f2 (table test-table [id=7846188897004a2282def2302f842c1a]) successfully deleted
I20260809 16:35:09.690442 12476 ts_tablet_manager.cc:1918] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 16:35:09.690524 12476 log.cc:1196] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/153b2946e0b54ee491a4659c5cf2d2f2
I20260809 16:35:09.690750 12476 ts_tablet_manager.cc:1939] T 153b2946e0b54ee491a4659c5cf2d2f2 P 379ddbf585384edcb71c6b3f4d97b84e: Deleting consensus metadata
I20260809 16:35:09.691051 11608 catalog_manager.cc:4841] TS 379ddbf585384edcb71c6b3f4d97b84e (127.31.15.195:35389): tablet 153b2946e0b54ee491a4659c5cf2d2f2 (table test-table [id=7846188897004a2282def2302f842c1a]) successfully deleted
W20260809 16:35:09.700227 11605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:43145 (error 108)
I20260809 16:35:09.700740 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:09.700573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43145 (local address 127.0.0.1:47376)
0809 16:35:09.700658 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:09.700698 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":43,"threads_started":1}
I20260809 16:35:09.700769 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 11954
W20260809 16:35:09.700868 11605 catalog_manager.cc:4584] TS 17985b0360fe40258cb28f8796082519 (127.31.15.193:43145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 153b2946e0b54ee491a4659c5cf2d2f2: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43145: connect: Connection refused (error 111)
I20260809 16:35:09.714759 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 12190
I20260809 16:35:09.721719 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 11582
2026-08-09T16:35:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2892 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[12581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[12581](info): set up 2 sockets
Aug 09 16:35:09 dist-test-slave-21b9 krb5kdc[12581](info): commencing operation
krb5kdc: starting...
W20260809 16:35:10.872823 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:10 dist-test-slave-21b9 krb5kdc[12581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:35:10.988014 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:43913
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:37493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:43913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:11.070430 12605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:11.070668 12605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:11.070703 12605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:11.071918 12605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:11.071966 12605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:11.071987 12605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:11.072007 12605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:11.072024 12605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:11.073557 12605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:43913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:43913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:11.073829 12605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:11.074057 12605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:11.078877 12616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:11.078884 12613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:11.078892 12614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:11.078974 12605 server_base.cc:1034] running on GCE node
I20260809 16:35:11.079435 12605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:11.079736 12605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:11.080914 12605 hybrid_clock.cc:648] HybridClock initialized: now 1786293311080891 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T16:35:11Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1357 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:35:11 dist-test-slave-21b9 krb5kdc[12629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:35:11 dist-test-slave-21b9 krb5kdc[12629](info): set up 2 sockets
Aug 09 16:35:11 dist-test-slave-21b9 krb5kdc[12629](info): commencing operation
krb5kdc: starting...
W20260809 16:35:12.239583 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:35:12.353891 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45071
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:12.434329 12649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:12.434540 12649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:12.434573 12649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:12.435719 12649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:12.435760 12649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:12.435781 12649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:12.435797 12649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:12.435813 12649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:12.437255 12649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:12.437492 12649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:12.437700 12649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:12.442181 12657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.442198 12663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.442188 12656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.442408 12649 server_base.cc:1034] running on GCE node
I20260809 16:35:12.442616 12649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:12.442857 12649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:12.443984 12649 hybrid_clock.cc:648] HybridClock initialized: now 1786293312443972 us; error 30 us; skew 500 ppm
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:12.452281 12649 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:35:12.452675 12649 webserver.cc:460] Webserver started at http://127.31.15.254:37261/ using document root <none> and password file <none>
I20260809 16:35:12.452877 12649 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:12.452926 12649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:12.453061 12649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:12.454102 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "f2e1206a17ce4dd3a7c650337562a0b2"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "30dea1844258aa966956ea7893cda430"
server_key_iv: "ff1f379ed41a9884d35963de05203ea8"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.454427 12649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "f2e1206a17ce4dd3a7c650337562a0b2"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "30dea1844258aa966956ea7893cda430"
server_key_iv: "ff1f379ed41a9884d35963de05203ea8"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.455838 12649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 16:35:12.456501 12674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.456660 12649 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:12.456748 12649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "f2e1206a17ce4dd3a7c650337562a0b2"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "30dea1844258aa966956ea7893cda430"
server_key_iv: "ff1f379ed41a9884d35963de05203ea8"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.456805 12649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:12.468133 12649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:12.469898 12649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:12.470003 12649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:12.473774 12649 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:45071
I20260809 16:35:12.473815 12754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:45071 every 8 connection(s)
I20260809 16:35:12.474133 12649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:12.474985 12755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:12.477250 12755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Bootstrap starting.
I20260809 16:35:12.477815 12755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:12.478082 12755 log.cc:824] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:12.478422 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 12649
I20260809 16:35:12.478528 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:12.478681 31807 external_mini_cluster.cc:1428] Setting key 1af48bae687280bc437cc052b9e78e1a
I20260809 16:35:12.478758 12755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: No bootstrap required, opened a new log
I20260809 16:35:12.480841 12755 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:12.481009 12755 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:12.481062 12755 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e1206a17ce4dd3a7c650337562a0b2, State: Initialized, Role: FOLLOWER
I20260809 16:35:12.481195 12755 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:12.481272 12755 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:12.481300 12755 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:12.481335 12755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:12.481976 12755 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:12.482090 12755 leader_election.cc:302] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e1206a17ce4dd3a7c650337562a0b2; no voters:
I20260809 16:35:12.482237 12755 leader_election.cc:288] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:12.482299 12764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:12.482453 12755 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:12.482497 12764 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 LEADER]: Becoming Leader. State: Replica: f2e1206a17ce4dd3a7c650337562a0b2, State: Running, Role: LEADER
I20260809 16:35:12.482605 12764 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:12.483009 12768 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } } }
I20260809 16:35:12.483163 12768 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:12.483349 12768 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e1206a17ce4dd3a7c650337562a0b2. Latest consensus state: current_term: 1 leader_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } } }
I20260809 16:35:12.483494 12768 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:12.483414 12778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:12.484078 12778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:12.486064 12778 catalog_manager.cc:1347] Generated new cluster ID: b1ebfd0c9a3848898899c68699608a1b
I20260809 16:35:12.486115 12778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:12.490665 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.479158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.31.15.254:45071)
0809 16:35:12.479354 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:12.479365 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:12.479400 (+ 35us) server_negotiation.cc:407] Connection header received
0809 16:35:12.480084 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:12.480094 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:12.480156 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:12.480292 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:12.480837 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.482451 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.483710 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.484300 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.484422 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.487111 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:12.487116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:12.487116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:12.487127 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:12.489140 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.489516 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.489519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.489519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.489617 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.489814 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.489815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.489815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.490098 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:12.490249 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:12.490344 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:35:12.490472 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260809 16:35:12.493696 12778 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:12.494398 12778 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:12.500669 12778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Generated new TSK 0
I20260809 16:35:12.500824 12778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:35:12.526367 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:12.607400 12799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:12.607591 12799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:12.607613 12799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:12.608743 12799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:12.608783 12799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:12.608808 12799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:12.610232 12799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:12.610448 12799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:12.610688 12799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:12.615504 12807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.615617 12799 server_base.cc:1034] running on GCE node
W20260809 16:35:12.615545 12809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.615495 12812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.615891 12799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:12.616097 12799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:12.617233 12799 hybrid_clock.cc:648] HybridClock initialized: now 1786293312617217 us; error 26 us; skew 500 ppm
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:12.619763 12799 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:35:12.620144 12799 webserver.cc:460] Webserver started at http://127.31.15.193:44881/ using document root <none> and password file <none>
I20260809 16:35:12.620311 12799 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:12.620350 12799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:12.620458 12799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:12.621415 12799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "0cd4785ff8064d51a0cdcac750b46593"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6b097fb9a34027aa50f4a751fa274a70"
server_key_iv: "40fae86ad2a22b89f1b7d74377211580"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.621711 12799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "0cd4785ff8064d51a0cdcac750b46593"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6b097fb9a34027aa50f4a751fa274a70"
server_key_iv: "40fae86ad2a22b89f1b7d74377211580"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.622942 12799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:12.623664 12827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.623816 12799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.623901 12799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "0cd4785ff8064d51a0cdcac750b46593"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6b097fb9a34027aa50f4a751fa274a70"
server_key_iv: "40fae86ad2a22b89f1b7d74377211580"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.623962 12799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:12.634873 12799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:12.636860 12799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:12.636991 12799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:12.637187 12799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:12.637466 12799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:12.637499 12799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.637526 12799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:12.637552 12799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.642887 12799 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:39435
I20260809 16:35:12.642966 12996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:39435 every 8 connection(s)
I20260809 16:35:12.643201 12799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:12.651079 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 12799
I20260809 16:35:12.651196 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:12.651413 31807 external_mini_cluster.cc:1428] Setting key 41235593896a0d807ade8d7bd00d605a
I20260809 16:35:12.651623 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.643885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:50345 (local address 127.31.15.254:45071)
0809 16:35:12.643931 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:12.643933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:12.644117 (+ 184us) server_negotiation.cc:407] Connection header received
0809 16:35:12.644573 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:12.644574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:12.644584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:12.644633 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:12.645508 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.646002 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.646926 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.647114 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.647155 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.649094 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:12.649096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:12.649096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:12.649102 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:12.650354 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.650673 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.650674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.650674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.650719 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.650934 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.650934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.650935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.651069 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:12.651131 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:12.651446 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 16:35:12.651535 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:35:12.651782 13001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.643851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:50345)
0809 16:35:12.644002 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:12.644036 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:12.644495 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:12.644685 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:12.644688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:12.645246 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:12.645435 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.645440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.646066 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.646067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.646835 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.646837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.647167 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.647168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.647265 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.647694 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:12.647701 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:12.649000 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:12.650414 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.650417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.650419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.650615 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.650764 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.650764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.650765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.650852 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.651132 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:12.651135 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:12.651306 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 16:35:12.651409 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:35:12.651562 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260809 16:35:12.652231 12997 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45071
I20260809 16:35:12.652328 12997 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:12.652520 12997 heartbeater.cc:511] Master 127.31.15.254:45071 requested a full tablet report, sending...
I20260809 16:35:12.653100 12699 ts_manager.cc:194] Registered new tserver with Master: 0cd4785ff8064d51a0cdcac750b46593 (127.31.15.193:39435)
I20260809 16:35:12.653883 12699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:50345
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:35:12.675082 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:12.754180 13007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:12.754377 13007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:12.754416 13007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:12.755553 13007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:12.755591 13007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:12.755625 13007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:12.757021 13007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:12.757256 13007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:12.757460 13007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:12.762059 13023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.762099 13017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.762136 13007 server_base.cc:1034] running on GCE node
W20260809 16:35:12.762102 13015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.762459 13007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:12.762684 13007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:12.763819 13007 hybrid_clock.cc:648] HybridClock initialized: now 1786293312763803 us; error 27 us; skew 500 ppm
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:12.766201 13007 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:35:12.766603 13007 webserver.cc:460] Webserver started at http://127.31.15.194:46259/ using document root <none> and password file <none>
I20260809 16:35:12.766800 13007 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:12.766839 13007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:12.766938 13007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:12.767890 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "3765afd1c36a4508951de4de6520fe33"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "a7f24927230b63cbfe7ef4174df71cea"
server_key_iv: "296d20f019ad262e39a0fddf0e50e615"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.768185 13007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "3765afd1c36a4508951de4de6520fe33"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "a7f24927230b63cbfe7ef4174df71cea"
server_key_iv: "296d20f019ad262e39a0fddf0e50e615"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.769343 13007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:12.769989 13032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.770141 13007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:12.770217 13007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "3765afd1c36a4508951de4de6520fe33"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "a7f24927230b63cbfe7ef4174df71cea"
server_key_iv: "296d20f019ad262e39a0fddf0e50e615"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.770279 13007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:12.782608 13007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:12.784475 13007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:12.784593 13007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:12.784789 13007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:12.785079 13007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:12.785116 13007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.785136 13007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:12.785147 13007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.790535 13007 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33699
I20260809 16:35:12.790587 13198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33699 every 8 connection(s)
I20260809 16:35:12.790879 13007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:12.799165 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.791579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:34105 (local address 127.31.15.254:45071)
0809 16:35:12.791645 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:12.791647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:12.791848 (+ 201us) server_negotiation.cc:407] Connection header received
0809 16:35:12.792304 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:12.792305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:12.792315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:12.792370 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:12.793256 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.793752 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.794689 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.794869 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.794919 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.797022 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:12.797024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:12.797024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:12.797032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:12.798160 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.798485 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.798486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.798486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.798530 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.798706 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.798707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.798707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.798799 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:12.798842 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:12.799043 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 16:35:12.799100 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:35:12.799361 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.791554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:34105)
0809 16:35:12.791722 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:12.791765 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:12.792227 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:12.792438 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:12.792442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:12.793004 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:12.793188 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.793192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.793829 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.793830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.794622 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.794624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.794923 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.794924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.795031 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.795554 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:12.795561 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:12.796926 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:12.798227 (+ 1301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.798232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.798234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.798427 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.798573 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.798573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.798574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.798666 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.798839 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:12.798842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:12.798944 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 16:35:12.799026 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:35:12.799135 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260809 16:35:12.799526 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13007
I20260809 16:35:12.799638 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:12.799847 31807 external_mini_cluster.cc:1428] Setting key 8dd8630d092149e1d454de3d67dd36c0
I20260809 16:35:12.799839 13200 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45071
I20260809 16:35:12.799917 13200 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:12.800104 13200 heartbeater.cc:511] Master 127.31.15.254:45071 requested a full tablet report, sending...
I20260809 16:35:12.800477 12699 ts_manager.cc:194] Registered new tserver with Master: 3765afd1c36a4508951de4de6520fe33 (127.31.15.194:33699)
I20260809 16:35:12.800824 12699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:34105
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:35:12.824128 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:12.904868 13207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:12.905076 13207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:12.905098 13207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:12.906286 13207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:12.906327 13207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:12.906352 13207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:12.907850 13207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:12.908082 13207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:12.908303 13207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:12.913113 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.913118 13214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:12.913133 13216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:12.913563 13207 server_base.cc:1034] running on GCE node
I20260809 16:35:12.913777 13207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:12.914032 13207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:12.915175 13207 hybrid_clock.cc:648] HybridClock initialized: now 1786293312915161 us; error 27 us; skew 500 ppm
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:12.917569 13207 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:35:12.917968 13207 webserver.cc:460] Webserver started at http://127.31.15.195:43811/ using document root <none> and password file <none>
I20260809 16:35:12.918200 13207 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:12.918247 13207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:12.918375 13207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:12.919493 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "8331b09f093c48569552e3cc1ffb8021"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6dba15dba96fad6ce74f46fe6d34442f"
server_key_iv: "4b07ba73db2f0e1870f58a27ae70be35"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.919832 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "8331b09f093c48569552e3cc1ffb8021"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6dba15dba96fad6ce74f46fe6d34442f"
server_key_iv: "4b07ba73db2f0e1870f58a27ae70be35"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.921085 13207 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:12.921746 13231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.921893 13207 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:12.921980 13207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "8331b09f093c48569552e3cc1ffb8021"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6dba15dba96fad6ce74f46fe6d34442f"
server_key_iv: "4b07ba73db2f0e1870f58a27ae70be35"
server_key_version: "encryptionkey@0"
I20260809 16:35:12.922047 13207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:12.948699 13207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:12.950546 13207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:12.950686 13207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:12.950932 13207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:12.951256 13207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:12.951335 13207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.951375 13207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:12.951399 13207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:12.956794 13207 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:33671
I20260809 16:35:12.956848 13403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:33671 every 8 connection(s)
I20260809 16:35:12.957159 13207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:12.959201 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13207
I20260809 16:35:12.959329 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:12.959515 31807 external_mini_cluster.cc:1428] Setting key 47903ff183458746cd656cd4471e6e05
Aug 09 16:35:12 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293312, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:12.965826 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.957848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:35043 (local address 127.31.15.254:45071)
0809 16:35:12.957912 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:12.957914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:12.958097 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:35:12.958616 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:12.958617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:12.958630 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:12.958681 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:12.959746 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.960367 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.961327 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.961530 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.961587 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.963569 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:12.963571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:12.963571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:12.963580 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:12.964601 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.964944 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.964945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.964946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.965010 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:12.965240 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:12.965240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:12.965241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:12.965346 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:12.965402 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:12.965658 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 16:35:12.965738 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:35:12.965979 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:12.957810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:35043)
0809 16:35:12.957980 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:12.958020 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:12.958492 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:12.958771 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:12.958776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:12.959419 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:12.959660 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.959665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.960455 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.960456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.961247 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:12.961249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:12.961600 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:12.961601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:12.961729 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:12.962123 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:12.962130 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:12.963472 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:12.964671 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.964677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.964679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.964881 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.965066 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:12.965067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:12.965067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:12.965171 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:12.965401 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:12.965407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:12.965543 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 16:35:12.965629 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:35:12.965775 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
I20260809 16:35:12.966399 13407 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45071
I20260809 16:35:12.966492 13407 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:12.966668 13407 heartbeater.cc:511] Master 127.31.15.254:45071 requested a full tablet report, sending...
I20260809 16:35:12.967003 12699 ts_manager.cc:194] Registered new tserver with Master: 8331b09f093c48569552e3cc1ffb8021 (127.31.15.195:33671)
I20260809 16:35:12.967402 12699 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:35043
I20260809 16:35:12.971047 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
Entry for principal oryx/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab.
WARNING: no policy specified for oryx/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
Entry for principal oryx/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab.
WARNING: no policy specified for oryx/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
Entry for principal oryx/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab.
I20260809 16:35:13.035415 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 12799
I20260809 16:35:13.046999 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13007
I20260809 16:35:13.062582 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13207
I20260809 16:35:13.077842 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 12649
I20260809 16:35:13.086074 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45071
--webserver_interface=127.31.15.254
--webserver_port=37261
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:13.167220 13421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:13.167464 13421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:13.167488 13421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:13.168664 13421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:13.168715 13421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:13.168730 13421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:13.168758 13421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:13.168784 13421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:13.170317 13421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=37261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:13.170569 13421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:13.170789 13421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:13.175361 13431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.175401 13421 server_base.cc:1034] running on GCE node
W20260809 16:35:13.175457 13429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.175370 13428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.175825 13421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:13.176092 13421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:13.177241 13421 hybrid_clock.cc:648] HybridClock initialized: now 1786293313177230 us; error 31 us; skew 500 ppm
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293313, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:13.179646 13421 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:35:13.180051 13421 webserver.cc:460] Webserver started at http://127.31.15.254:37261/ using document root <none> and password file <none>
I20260809 16:35:13.180256 13421 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:13.180321 13421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:13.181633 13421 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:13.182246 13443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.182407 13421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:13.182492 13421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "f2e1206a17ce4dd3a7c650337562a0b2"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "30dea1844258aa966956ea7893cda430"
server_key_iv: "ff1f379ed41a9884d35963de05203ea8"
server_key_version: "encryptionkey@0"
I20260809 16:35:13.182803 13421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:13.193862 13421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:13.195976 13421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:13.196107 13421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:13.200064 13421 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:45071
I20260809 16:35:13.200112 13529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:45071 every 8 connection(s)
I20260809 16:35:13.200453 13421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:13.200776 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13421
I20260809 16:35:13.201041 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:39435
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:45071
--webserver_port=44881
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:35:13.201999 13532 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:35:13.202764 13532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Bootstrap starting.
I20260809 16:35:13.211292 13532 log.cc:824] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:13.213048 13532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:35:13.213323 13532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Bootstrap complete.
I20260809 16:35:13.215463 13532 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:13.215696 13532 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e1206a17ce4dd3a7c650337562a0b2, State: Initialized, Role: FOLLOWER
I20260809 16:35:13.215842 13532 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:13.215906 13532 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:13.215945 13532 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:13.215991 13532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:35:13.216952 13532 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:13.217043 13532 leader_election.cc:302] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2e1206a17ce4dd3a7c650337562a0b2; no voters:
I20260809 16:35:13.217202 13532 leader_election.cc:288] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:35:13.217257 13538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:35:13.217379 13538 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [term 2 LEADER]: Becoming Leader. State: Replica: f2e1206a17ce4dd3a7c650337562a0b2, State: Running, Role: LEADER
I20260809 16:35:13.217474 13532 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:13.217463 13538 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } }
I20260809 16:35:13.217790 13538 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2e1206a17ce4dd3a7c650337562a0b2. Latest consensus state: current_term: 2 leader_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } } }
I20260809 16:35:13.217882 13538 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:13.218183 13541 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2e1206a17ce4dd3a7c650337562a0b2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45071 } } }
I20260809 16:35:13.218274 13541 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:13.218570 13548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:13.219094 13548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:13.219259 13548 catalog_manager.cc:1259] Loaded cluster ID: b1ebfd0c9a3848898899c68699608a1b
I20260809 16:35:13.219488 13548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:13.220285 13548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:13.220448 13548 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f2e1206a17ce4dd3a7c650337562a0b2: Loaded TSK: 0
I20260809 16:35:13.220641 13548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:35:13.311746 13534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:13.312026 13534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:13.312067 13534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:13.313753 13534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:13.313827 13534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:13.313874 13534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:13.315975 13534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.193.keytab
--principal=oryx/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:39435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=44881
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:13.316231 13534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:13.316583 13534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:13.322026 13576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.322146 13534 server_base.cc:1034] running on GCE node
W20260809 16:35:13.322037 13578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.322311 13575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.322587 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:13.322868 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:13.324048 13534 hybrid_clock.cc:648] HybridClock initialized: now 1786293313324027 us; error 33 us; skew 500 ppm
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293313, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:13.327026 13534 init.cc:377] Logged in from keytab as oryx/127.31.15.193@KRBTEST.COM (short username oryx)
I20260809 16:35:13.327473 13534 webserver.cc:460] Webserver started at http://127.31.15.193:44881/ using document root <none> and password file <none>
I20260809 16:35:13.327675 13534 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:13.327740 13534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:13.329088 13534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:13.329725 13587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.329881 13534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:13.329968 13534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "0cd4785ff8064d51a0cdcac750b46593"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6b097fb9a34027aa50f4a751fa274a70"
server_key_iv: "40fae86ad2a22b89f1b7d74377211580"
server_key_version: "encryptionkey@0"
I20260809 16:35:13.330298 13534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:13.350739 13534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:13.352885 13534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:13.353031 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:13.353262 13534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:13.353560 13534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:13.353595 13534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.353624 13534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:13.353646 13534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.359290 13534 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:39435
I20260809 16:35:13.359354 13766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:39435 every 8 connection(s)
I20260809 16:35:13.359663 13534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.367060 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13534
I20260809 16:35:13.367353 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:33699
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:45071
--webserver_port=46259
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:35:13.367954 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.360332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:35683)
0809 16:35:13.360500 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.360540 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.361047 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.361450 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.361454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.362026 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.362210 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.362216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.363324 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.363325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.364152 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.364155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.364710 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.364711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.364834 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.365353 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.365360 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.367755 (+ 2395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
W20260809 16:35:13.368170 13768 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:45071 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.15.254:45071: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
W20260809 16:35:13.368410 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.360433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35683 (local address 127.31.15.254:45071)
0809 16:35:13.360637 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.360646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.360686 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:35:13.361241 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.361246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.361288 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.361405 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.362329 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.363208 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.364235 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.364631 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.364725 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.367930 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:35683: BlockingRecv error: recv got EOF from 127.31.15.193:35683 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.374223 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.368358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:44755)
0809 16:35:13.368522 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.368532 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.368569 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.369408 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.369409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.369559 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.369612 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.369614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.370414 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.370415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.371142 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.371146 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.371597 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.371598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.371665 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.372096 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.372102 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.374154 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
W20260809 16:35:13.374307 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.369233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:44755 (local address 127.31.15.254:45071)
0809 16:35:13.369287 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.369289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.369296 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:13.369317 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.369318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.369330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.369388 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.369682 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.370338 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.371233 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.371520 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.371597 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.374254 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:44755: BlockingRecv error: recv got EOF from 127.31.15.193:44755 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.379624 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.374748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:36569)
0809 16:35:13.374792 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.374802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.374834 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.375003 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.375004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.375136 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.375168 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.375170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.376045 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.376046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.376792 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.376795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.377205 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.377206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.377258 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.377668 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.377673 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.379565 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:35:13.379715 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.374813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:36569 (local address 127.31.15.254:45071)
0809 16:35:13.374895 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.374897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.374904 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:13.374921 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.374922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.374932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.374977 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.375242 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.375970 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.376875 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.377137 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.377192 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.379652 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:36569: BlockingRecv error: recv got EOF from 127.31.15.193:36569 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 16:35:13.379992 13768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:35:13.451035 13776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:13.451231 13776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:13.451251 13776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:13.452394 13776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:13.452435 13776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:13.452459 13776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:13.453850 13776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.194.keytab
--principal=oryx/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:33699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=46259
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:13.454073 13776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:13.454280 13776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:13.458882 13782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.458891 13783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.458906 13787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.459198 13776 server_base.cc:1034] running on GCE node
I20260809 16:35:13.459419 13776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:13.459646 13776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:13.460801 13776 hybrid_clock.cc:648] HybridClock initialized: now 1786293313460788 us; error 29 us; skew 500 ppm
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293313, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:13.463251 13776 init.cc:377] Logged in from keytab as oryx/127.31.15.194@KRBTEST.COM (short username oryx)
I20260809 16:35:13.463687 13776 webserver.cc:460] Webserver started at http://127.31.15.194:46259/ using document root <none> and password file <none>
I20260809 16:35:13.463889 13776 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:13.463963 13776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:13.465219 13776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:13.465725 13795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.465874 13776 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.465968 13776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "3765afd1c36a4508951de4de6520fe33"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "a7f24927230b63cbfe7ef4174df71cea"
server_key_iv: "296d20f019ad262e39a0fddf0e50e615"
server_key_version: "encryptionkey@0"
I20260809 16:35:13.466284 13776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:13.483919 13776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:13.486136 13776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:13.486268 13776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:13.486488 13776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:13.486785 13776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:13.486819 13776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.486845 13776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:13.486862 13776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.492434 13776 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33699
I20260809 16:35:13.492506 13989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33699 every 8 connection(s)
I20260809 16:35:13.492808 13776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:35:13.493651 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13776
I20260809 16:35:13.493918 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:33671
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:45071
--webserver_port=43811
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.500229 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.493448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:57743)
0809 16:35:13.493622 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.493657 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.494228 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.494468 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.494472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.495022 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.495207 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.495212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.495944 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.495945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.497025 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.497027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.497443 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.497443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.497558 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.498031 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.498039 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.500087 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
W20260809 16:35:13.500257 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.493466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:57743 (local address 127.31.15.254:45071)
0809 16:35:13.493515 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.493517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.493736 (+ 219us) server_negotiation.cc:407] Connection header received
0809 16:35:13.494326 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.494328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.494347 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.494415 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.495311 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.495862 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.497113 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.497370 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.497443 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.500195 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:57743: BlockingRecv error: recv got EOF from 127.31.15.194:57743 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 16:35:13.500459 13991 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:45071 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.15.254:45071: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.504940 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.500688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:48637)
0809 16:35:13.500724 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.500735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.500776 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.500901 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.500902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.501003 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.501053 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.501054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.501878 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.501879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.502391 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.502393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.502806 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.502807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.502853 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.503245 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.503249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.504895 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260809 16:35:13.505069 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.500718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:48637 (local address 127.31.15.254:45071)
0809 16:35:13.500785 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.500787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.500797 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:35:13.500824 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.500825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.500835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.500889 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.501112 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.501782 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.502471 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.502743 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.502812 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.504999 (+ 2187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:48637: BlockingRecv error: recv got EOF from 127.31.15.194:48637 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.509341 13996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.505170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:59775)
0809 16:35:13.505286 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.505296 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.505342 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.505458 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.505459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.505557 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.505583 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.505584 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.506414 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.506415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.506884 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.506886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.507335 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.507336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.507370 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.507731 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.507734 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.509298 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
W20260809 16:35:13.509441 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.505225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:59775 (local address 127.31.15.254:45071)
0809 16:35:13.505255 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.505257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.505342 (+ 85us) server_negotiation.cc:407] Connection header received
0809 16:35:13.505395 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.505395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.505404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.505449 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.505635 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.506351 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.506946 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.507236 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.507327 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.509387 (+ 2060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:59775: BlockingRecv error: recv got EOF from 127.31.15.194:59775 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 16:35:13.509454 13991 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:35:13.577847 13998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:13.578028 13998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:13.578049 13998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:13.579156 13998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:13.579195 13998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:13.579218 13998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:13.580652 13998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/krb5kdc/oryx_127.31.15.195.keytab
--principal=oryx/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:33671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=43811
--tserver_master_addrs=127.31.15.254:45071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:13.580858 13998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:13.581058 13998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:13.585623 14008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.585642 14006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:13.585645 14005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:13.585916 13998 server_base.cc:1034] running on GCE node
I20260809 16:35:13.586143 13998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:13.586330 13998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:13.587459 13998 hybrid_clock.cc:648] HybridClock initialized: now 1786293313587448 us; error 28 us; skew 500 ppm
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293313, etypes {rep=17 tkt=17 ses=17}, oryx/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:13.589759 13998 init.cc:377] Logged in from keytab as oryx/127.31.15.195@KRBTEST.COM (short username oryx)
I20260809 16:35:13.590150 13998 webserver.cc:460] Webserver started at http://127.31.15.195:43811/ using document root <none> and password file <none>
I20260809 16:35:13.590360 13998 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:13.590420 13998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:13.591689 13998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:13.592255 14024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.592417 13998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:13.592490 13998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "8331b09f093c48569552e3cc1ffb8021"
format_stamp: "Formatted at 2026-08-09 16:35:12 on dist-test-slave-21b9"
server_key: "6dba15dba96fad6ce74f46fe6d34442f"
server_key_iv: "4b07ba73db2f0e1870f58a27ae70be35"
server_key_version: "encryptionkey@0"
I20260809 16:35:13.592780 13998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:13.606294 13998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:13.608076 13998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:13.608201 13998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:13.608398 13998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:13.608675 13998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:13.608708 13998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.608734 13998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:13.608758 13998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:13.614112 13998 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:33671
I20260809 16:35:13.614172 14199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:33671 every 8 connection(s)
I20260809 16:35:13.614470 13998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:13.619699 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 13998
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.621457 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.615159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:36973)
0809 16:35:13.615333 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.615371 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.615827 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.616029 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.616033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.616577 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.616756 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.616761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.617461 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.617461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.618320 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.618323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.618683 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.618684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.618814 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.619220 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.619228 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.621325 (+ 2097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260809 16:35:13.621464 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.615195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:36973 (local address 127.31.15.254:45071)
0809 16:35:13.615262 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.615263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.615448 (+ 185us) server_negotiation.cc:407] Connection header received
0809 16:35:13.615901 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.615901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.615911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.615973 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.616862 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.617372 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.618386 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.618605 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.618669 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.621405 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:36973: BlockingRecv error: recv got EOF from 127.31.15.195:36973 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 16:35:13.621708 14200 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:45071 (0 consecutive failures): Not authorized: Failed to ping master at 127.31.15.254:45071: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.625917 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.621916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:54751)
0809 16:35:13.621953 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.621963 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.621993 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.622124 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.622125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.622234 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.622272 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.622274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.622840 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.622841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.623601 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.623604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.623908 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.623909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.623972 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.624381 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.624385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.625859 (+ 1474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:35:13.625991 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.621978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:54751 (local address 127.31.15.254:45071)
0809 16:35:13.622036 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.622038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.622044 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:13.622061 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.622062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.622070 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.622116 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.622335 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.622772 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.623673 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.623841 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.623899 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.625940 (+ 2041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:54751: BlockingRecv error: recv got EOF from 127.31.15.195:54751 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:35:13.626483 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.620174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.31.15.254:45071)
0809 16:35:13.620278 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.620279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.620287 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:13.620304 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.620305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.620315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.620365 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.620947 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.621451 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.622323 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.622530 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.622583 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.623850 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:13.623853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:13.623854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:13.623864 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:13.625196 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:13.625501 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:13.625504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:13.625504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:13.625590 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:13.625776 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:13.625777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:13.625777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:13.625998 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:13.626129 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:13.626239 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:35:13.626352 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:13 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:13.629902 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:13.626365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:42671)
0809 16:35:13.626403 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:13.626411 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:13.626441 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:13.626554 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:13.626555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:13.626662 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:13.626685 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.626686 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.627443 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.627443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.627906 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:13.627907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.628174 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.628175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:13.628207 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.628503 (+ 296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:13.628506 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:13.629848 (+ 1342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:35:13.629966 13771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:13.626401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:42671 (local address 127.31.15.254:45071)
0809 16:35:13.626459 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:13.626461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:13.626467 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:13.626485 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:13.626485 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:13.626493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:13.626550 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:13.626759 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.627387 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.627963 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:13.628130 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:13.628170 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:13.629922 (+ 1752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:42671: BlockingRecv error: recv got EOF from 127.31.15.195:42671 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 16:35:13.630069 14200 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:14.384986 14209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:14.380350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:49387)
0809 16:35:14.380470 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:14.380480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:14.380528 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:14.380655 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:14.380656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:14.380816 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:14.380864 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.380866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.381551 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.381553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.382176 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.382180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.382591 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.382592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.382630 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.383080 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:14.383085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:14.384899 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
W20260809 16:35:14.385032 14210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:14.380424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:49387 (local address 127.31.15.254:45071)
0809 16:35:14.380542 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:14.380543 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:14.380552 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:14.380571 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:14.380572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:14.380581 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:14.380639 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:14.380926 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.381424 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.382279 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.382494 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.382572 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.384971 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:49387: BlockingRecv error: recv got EOF from 127.31.15.193:49387 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:14.514160 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:14.509807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:40103)
0809 16:35:14.509934 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:14.509948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:14.510029 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:14.510152 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:14.510153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:14.510325 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:14.510370 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.510372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.511005 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.511006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.511574 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.511576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.511894 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.511896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.511947 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.512376 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:14.512380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:14.514082 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
W20260809 16:35:14.514221 14210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:14.509899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40103 (local address 127.31.15.254:45071)
0809 16:35:14.509957 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:14.509959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:14.510020 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:35:14.510080 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:14.510081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:14.510093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:14.510139 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:14.510441 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.510935 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.511651 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.511837 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.511890 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.514171 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:40103: BlockingRecv error: recv got EOF from 127.31.15.194:40103 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:14 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:14.634704 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:14.630434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:38419)
0809 16:35:14.630580 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:14.630592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:14.630649 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:14.630759 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:14.630760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:14.630920 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:14.630970 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.630971 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.631611 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.631612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.632148 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:14.632150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.632498 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.632498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:14.632540 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.632922 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:14.632926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:14.634617 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
W20260809 16:35:14.634765 14210 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:14.630493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:38419 (local address 127.31.15.254:45071)
0809 16:35:14.630550 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:14.630552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:14.630642 (+ 90us) server_negotiation.cc:407] Connection header received
0809 16:35:14.630696 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:14.630696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:14.630708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:14.630746 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:14.631018 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.631544 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.632225 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:14.632439 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:14.632482 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:14.634713 (+ 2231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:38419: BlockingRecv error: recv got EOF from 127.31.15.195:38419 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:15.389899 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:15.385532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:46883)
0809 16:35:15.385671 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:15.385681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:15.385776 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:15.385933 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:15.385934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:15.386144 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:15.386209 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.386212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.386865 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.386866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.387416 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.387418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.387709 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.387710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.387745 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.388183 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:15.388188 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:15.389843 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
W20260809 16:35:15.390002 14215 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:15.385595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:46883 (local address 127.31.15.254:45071)
0809 16:35:15.385696 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:15.385698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:15.385756 (+ 58us) server_negotiation.cc:407] Connection header received
0809 16:35:15.385846 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:15.385847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:15.385859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:15.385919 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:15.386278 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.386807 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.387469 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.387662 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.387706 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.389926 (+ 2220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:46883: BlockingRecv error: recv got EOF from 127.31.15.193:46883 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:15.518893 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:15.514600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:44009)
0809 16:35:15.514745 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:15.514756 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:15.514824 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:15.514929 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:15.514929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:15.515090 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:15.515139 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.515141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.515756 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.515757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.516270 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.516271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.516545 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.516546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.516581 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.517078 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:15.517084 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:15.518833 (+ 1749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260809 16:35:15.518996 14215 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:15.514639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:44009 (local address 127.31.15.254:45071)
0809 16:35:15.514683 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:15.514685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:15.514831 (+ 146us) server_negotiation.cc:407] Connection header received
0809 16:35:15.514859 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:15.514859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:15.514871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:15.514916 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:15.515190 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.515699 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.516326 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.516500 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.516541 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.518925 (+ 2384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:44009: BlockingRecv error: recv got EOF from 127.31.15.194:44009 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:15 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:15.639523 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:15.635225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:49739)
0809 16:35:15.635383 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:15.635397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:15.635464 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:15.635594 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:15.635595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:15.635760 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:15.635808 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.635810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.636426 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.636427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.636950 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:15.636952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.637242 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.637243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:15.637286 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.637702 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:15.637705 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:15.639435 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
W20260809 16:35:15.639578 14215 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:15.635355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:49739 (local address 127.31.15.254:45071)
0809 16:35:15.635409 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:15.635411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:15.635448 (+ 37us) server_negotiation.cc:407] Connection header received
0809 16:35:15.635517 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:15.635517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:15.635531 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:15.635584 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:15.635857 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.636356 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.637006 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:15.637188 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:15.637236 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:15.639524 (+ 2288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:49739: BlockingRecv error: recv got EOF from 127.31.15.195:49739 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:16.398562 14220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:16.390323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:50733)
0809 16:35:16.390461 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:16.390482 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:16.390544 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:16.390656 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:16.390656 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:16.390820 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:16.390869 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.390871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.391563 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.391564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.392098 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.392099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.392400 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.392400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.392438 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.396760 (+ 4322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:16.396765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:16.398492 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
W20260809 16:35:16.398684 14221 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:16.390390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:50733 (local address 127.31.15.254:45071)
0809 16:35:16.390548 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:16.390549 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:16.390559 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:35:16.390580 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:16.390580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:16.390590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:16.390645 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:16.390920 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.391467 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.392165 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.392352 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.392396 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.398607 (+ 6211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:50733: BlockingRecv error: recv got EOF from 127.31.15.193:50733 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:16.523741 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:16.519342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:42615)
0809 16:35:16.519490 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:16.519502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:16.519568 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:16.519707 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:16.519708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:16.519884 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:16.519933 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.519935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.520586 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.520588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.521117 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.521119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.521458 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.521459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.521515 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.521941 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:16.521946 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:16.523663 (+ 1717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260809 16:35:16.523826 14221 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:16.519392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:42615 (local address 127.31.15.254:45071)
0809 16:35:16.519462 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:16.519464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:16.519562 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:35:16.519633 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:16.519633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:16.519647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:16.519696 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:16.520010 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.520515 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.521198 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.521392 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.521447 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.523772 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:42615: BlockingRecv error: recv got EOF from 127.31.15.194:42615 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:16 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:16.644354 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:16.640029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:56673)
0809 16:35:16.640173 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:16.640186 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:16.640250 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:16.640391 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:16.640392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:16.640561 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:16.640612 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.640614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.641215 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.641216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.641725 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:16.641727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.642055 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.642055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:16.642092 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.642482 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:16.642486 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:16.644294 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
W20260809 16:35:16.644433 14221 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:16.640087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56673 (local address 127.31.15.254:45071)
0809 16:35:16.640139 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:16.640141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:16.640291 (+ 150us) server_negotiation.cc:407] Connection header received
0809 16:35:16.640317 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:16.640317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:16.640334 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:16.640376 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:16.640666 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.641164 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.641799 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:16.641997 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:16.642051 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:16.644377 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:56673: BlockingRecv error: recv got EOF from 127.31.15.195:56673 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:17.403355 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:17.399006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:34287)
0809 16:35:17.399135 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:17.399148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:17.399202 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:17.399350 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:17.399351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:17.399514 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:17.399572 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.399574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.400228 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.400229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.400758 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.400760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.401050 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.401051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.401085 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.401503 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:17.401508 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:17.403245 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
W20260809 16:35:17.403437 14228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:17.399055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:34287 (local address 127.31.15.254:45071)
0809 16:35:17.399176 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:17.399178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:17.399187 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:17.399244 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:17.399245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:17.399256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:17.399337 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:17.399657 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.400163 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.400812 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.400994 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.401046 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.403373 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:34287: BlockingRecv error: recv got EOF from 127.31.15.193:34287 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:17.532208 14231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:17.524215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:53459)
0809 16:35:17.524350 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:17.524362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:17.524441 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:17.524587 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:17.524588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:17.524758 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:17.524806 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.524807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.525457 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.525458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.526064 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.526068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.526379 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.526380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.526430 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.530433 (+ 4003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:17.530438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:17.532146 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
W20260809 16:35:17.532311 14228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:17.524294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:53459 (local address 127.31.15.254:45071)
0809 16:35:17.524357 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:17.524359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:17.524429 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:35:17.524504 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:17.524505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:17.524518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:17.524571 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:17.524879 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.525384 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.526124 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.526320 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.526377 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.532250 (+ 5873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:53459: BlockingRecv error: recv got EOF from 127.31.15.194:53459 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:17 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:17.649165 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:17.644809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:35809)
0809 16:35:17.644932 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:17.644947 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:17.645012 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:17.645158 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:17.645159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:17.645329 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:17.645377 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.645379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.646017 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.646019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.646536 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:17.646538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.646872 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.646873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:17.646929 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.647431 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:17.647436 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:17.649092 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
W20260809 16:35:17.649238 14228 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:17.644901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:35809 (local address 127.31.15.254:45071)
0809 16:35:17.644967 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:17.644969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:17.644992 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:35:17.645066 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:17.645067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:17.645093 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:17.645144 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:17.645449 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.645947 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.646614 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:17.646812 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:17.646867 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:17.649182 (+ 2315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:35809: BlockingRecv error: recv got EOF from 127.31.15.195:35809 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:18.408434 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:18.403889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:35353)
0809 16:35:18.404094 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:18.404105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:18.404162 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:18.404293 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:18.404294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:18.404467 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:18.404534 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.404537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.405191 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.405192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.405731 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.405733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.406064 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.406066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.406131 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.406565 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:18.406571 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:18.408376 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
W20260809 16:35:18.408885 14243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:18.403925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35353 (local address 127.31.15.254:45071)
0809 16:35:18.404067 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:18.404069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:18.404159 (+ 90us) server_negotiation.cc:407] Connection header received
0809 16:35:18.404221 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:18.404221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:18.404234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:18.404287 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:18.404609 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.405120 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.405801 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.405999 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.406058 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.408687 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:35353: BlockingRecv error: recv got EOF from 127.31.15.193:35353 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:18.537197 14246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:18.532628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:40151)
0809 16:35:18.532762 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:18.532775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:18.532855 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:18.532987 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:18.532988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:18.533156 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:18.533205 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.533206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.533806 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.533806 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.534389 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.534390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.534730 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.534731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.534769 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.535181 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:18.535186 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:18.537135 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
W20260809 16:35:18.537274 14243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:18.532743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40151 (local address 127.31.15.254:45071)
0809 16:35:18.532796 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:18.532798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:18.532860 (+ 62us) server_negotiation.cc:407] Connection header received
0809 16:35:18.532907 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:18.532908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:18.532919 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:18.532973 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:18.533256 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.533751 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.534449 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.534644 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.534726 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.537219 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:40151: BlockingRecv error: recv got EOF from 127.31.15.194:40151 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:18 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:18.653915 14247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:18.649610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:34211)
0809 16:35:18.649730 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:18.649741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:18.649787 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:18.649909 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:18.649910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:18.650073 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:18.650121 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.650122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.650772 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.650773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.651317 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:18.651319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.651687 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.651688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:18.651737 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.652151 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:18.652155 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:18.653857 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
W20260809 16:35:18.654017 14243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:18.649714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:34211 (local address 127.31.15.254:45071)
0809 16:35:18.649766 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:18.649768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:18.649778 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:35:18.649831 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:18.649832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:18.649845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:18.649903 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:18.650193 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.650710 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.651399 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:18.651610 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:18.651667 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:18.653948 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:34211: BlockingRecv error: recv got EOF from 127.31.15.195:34211 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:19.413585 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:19.409032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:46689)
0809 16:35:19.409219 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:19.409232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:19.409316 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:19.409494 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:19.409496 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:19.409726 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:19.409789 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.409792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.410449 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.410450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.410989 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.410991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.411371 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.411372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.411422 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.411906 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:19.411911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:19.413514 (+ 1603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
W20260809 16:35:19.413678 14249 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:19.409108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:46689 (local address 127.31.15.254:45071)
0809 16:35:19.409247 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:19.409249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:19.409296 (+ 47us) server_negotiation.cc:407] Connection header received
0809 16:35:19.409382 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:19.409383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:19.409398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:19.409474 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:19.409866 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.410389 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.411056 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.411251 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.411337 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.413607 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:46689: BlockingRecv error: recv got EOF from 127.31.15.193:46689 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:19.542227 14251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:19.537638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:36377)
0809 16:35:19.537799 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:19.537813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:19.537889 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:19.538029 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:19.538030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:19.538213 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:19.538263 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.538265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.538957 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.538959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.539543 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.539546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.539887 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.539888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.539939 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.540408 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:19.540413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:19.542166 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260809 16:35:19.542338 14249 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:19.537710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36377 (local address 127.31.15.254:45071)
0809 16:35:19.537803 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:19.537805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:19.537883 (+ 78us) server_negotiation.cc:407] Connection header received
0809 16:35:19.537953 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:19.537954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:19.537968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:19.538025 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:19.538336 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.538860 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.539622 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.539821 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.539895 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.542266 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:36377: BlockingRecv error: recv got EOF from 127.31.15.194:36377 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:19 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:19.658658 14253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:19.654352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:56821)
0809 16:35:19.654512 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:19.654523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:19.654583 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:19.654697 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:19.654698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:19.654875 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:19.654924 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.654926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.655537 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.655537 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.656053 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:19.656054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.656322 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.656322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:19.656358 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.656821 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:19.656827 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:19.658599 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
W20260809 16:35:19.658754 14249 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:19.654446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56821 (local address 127.31.15.254:45071)
0809 16:35:19.654507 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:19.654509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:19.654579 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:35:19.654623 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:19.654624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:19.654635 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:19.654686 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:19.654974 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.655487 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.656107 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:19.656277 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:19.656319 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:19.658691 (+ 2372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:56821: BlockingRecv error: recv got EOF from 127.31.15.195:56821 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:20.420208 14254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:20.414027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:37753)
0809 16:35:20.415991 (+ 1964us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:20.416005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:20.416077 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:20.416214 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:20.416215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:20.416390 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:20.416436 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.416438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.417121 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.417122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.417642 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.417644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.418010 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.418011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.418067 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.418476 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:20.418480 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:20.420131 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1942,"thread_start_us":1892,"threads_started":1}
W20260809 16:35:20.420292 14256 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:20.414127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:37753 (local address 127.31.15.254:45071)
0809 16:35:20.415995 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:20.415998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:20.416067 (+ 69us) server_negotiation.cc:407] Connection header received
0809 16:35:20.416131 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:20.416132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:20.416145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:20.416204 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:20.416507 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.417039 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.417729 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.417948 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.418005 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.420229 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:37753: BlockingRecv error: recv got EOF from 127.31.15.193:37753 (error 108)
Metrics: {"server-negotiator.queue_time_us":1839,"thread_start_us":1791,"threads_started":1}
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:20.547067 14257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:20.542685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:36465)
0809 16:35:20.542841 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:20.542851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:20.542905 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:20.543034 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:20.543035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:20.543205 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:20.543255 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.543256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.543904 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.543905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.544448 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.544451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.544742 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.544742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.544779 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.545227 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:20.545232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:20.546997 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
W20260809 16:35:20.547154 14256 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:20.542779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36465 (local address 127.31.15.254:45071)
0809 16:35:20.542838 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:20.542840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:20.542899 (+ 59us) server_negotiation.cc:407] Connection header received
0809 16:35:20.542954 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:20.542955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:20.542966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:20.543023 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:20.543321 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.543820 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.544519 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.544695 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.544736 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.547094 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:36465: BlockingRecv error: recv got EOF from 127.31.15.194:36465 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:20 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:20.663514 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:20.659122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:45891)
0809 16:35:20.659290 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:20.659301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:20.659357 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:20.659486 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:20.659487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:20.659645 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:20.659693 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.659695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.660295 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.660296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.660831 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:20.660833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.661152 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.661152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:20.661189 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.661597 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:20.661602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:20.663433 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
W20260809 16:35:20.663570 14256 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:20.659185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:45891 (local address 127.31.15.254:45071)
0809 16:35:20.659232 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:20.659234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:20.659352 (+ 118us) server_negotiation.cc:407] Connection header received
0809 16:35:20.659403 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:20.659404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:20.659416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:20.659477 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:20.659740 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.660235 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.660908 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:20.661098 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:20.661148 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:20.663521 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:45891: BlockingRecv error: recv got EOF from 127.31.15.195:45891 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:21.424935 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:21.420696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:36559)
0809 16:35:21.420821 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:21.420832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:21.420884 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:21.421017 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:21.421017 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:21.421183 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:21.421229 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.421231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.421870 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.421870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.422385 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.422386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.422679 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.422680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.422715 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.423129 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:21.423133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:21.424867 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
W20260809 16:35:21.425035 14263 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:21.420796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:36559 (local address 127.31.15.254:45071)
0809 16:35:21.420913 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:21.420915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:21.420921 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:35:21.420940 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:21.420940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:21.420950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:21.421008 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:21.421301 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.421814 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.422443 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.422631 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.422677 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.424967 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:36559: BlockingRecv error: recv got EOF from 127.31.15.193:36559 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":54,"threads_started":1}
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:21.552146 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:21.547551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:58761)
0809 16:35:21.547703 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:21.547714 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:21.547780 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:21.547909 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:21.547910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:21.548094 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:21.548139 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.548141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.549017 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.549018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.549546 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.549548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.549870 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.549871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.549923 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.550351 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:21.550355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:21.552069 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
W20260809 16:35:21.552224 14263 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:21.547615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:58761 (local address 127.31.15.254:45071)
0809 16:35:21.547670 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:21.547672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:21.547772 (+ 100us) server_negotiation.cc:407] Connection header received
0809 16:35:21.547835 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:21.547836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:21.547849 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:21.547897 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:21.548217 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.548937 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.549622 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.549810 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.549865 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.552172 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:58761: BlockingRecv error: recv got EOF from 127.31.15.194:58761 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:21 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:21.668440 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:21.664013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:38811)
0809 16:35:21.664189 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:21.664200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:21.664264 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:21.664387 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:21.664388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:21.664568 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:21.664622 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.664625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.665355 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.665356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.665880 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:21.665881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.666163 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.666163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:21.666201 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.666642 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:21.666647 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:21.668381 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260809 16:35:21.668591 14263 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:21.664096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:38811 (local address 127.31.15.254:45071)
0809 16:35:21.664168 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:21.664170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:21.664254 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:35:21.664304 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:21.664305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:21.664319 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:21.664365 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:21.664679 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.665289 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.665936 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:21.666117 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:21.666159 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:21.668524 (+ 2365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:38811: BlockingRecv error: recv got EOF from 127.31.15.195:38811 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:22.429663 14272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:22.425400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:37155)
0809 16:35:22.425599 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:22.425610 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:22.425665 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:22.425796 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:22.425797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:22.425955 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:22.426004 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.426006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.426621 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.426622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.427146 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.427148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.427476 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.427477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.427523 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.427913 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:22.427917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:22.429605 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
W20260809 16:35:22.429764 14273 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:22.425491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:37155 (local address 127.31.15.254:45071)
0809 16:35:22.425618 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:22.425620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:22.425653 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:35:22.425712 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:22.425712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:22.425725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:22.425787 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:22.426055 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.426545 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.427202 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.427426 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.427475 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.429694 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:37155: BlockingRecv error: recv got EOF from 127.31.15.193:37155 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:22.559142 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:22.552620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:35045)
0809 16:35:22.552767 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:22.552777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:22.552836 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:22.552946 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:22.552947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:22.553124 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:22.553173 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.553174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.553771 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.553772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.554290 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.554291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.554559 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.554560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.554595 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.557420 (+ 2825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:22.557425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:22.559074 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
W20260809 16:35:22.559230 14273 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:22.552692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:35045 (local address 127.31.15.254:45071)
0809 16:35:22.552754 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:22.552756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:22.552831 (+ 75us) server_negotiation.cc:407] Connection header received
0809 16:35:22.552876 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:22.552877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:22.552889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:22.552935 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:22.553223 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.553708 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.554344 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.554515 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.554556 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.559171 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:35045: BlockingRecv error: recv got EOF from 127.31.15.194:35045 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:22 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:22.673096 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:22.668841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:56817)
0809 16:35:22.668979 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:22.668996 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:22.669051 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:22.669169 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:22.669170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:22.669327 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:22.669379 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.669380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.670036 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.670037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.670555 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:22.670557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.670830 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.670830 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:22.670866 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.671288 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:22.671292 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:22.673003 (+ 1711us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
W20260809 16:35:22.673161 14273 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:22.668893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56817 (local address 127.31.15.254:45071)
0809 16:35:22.668948 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:22.668950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:22.669046 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:35:22.669091 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:22.669091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:22.669103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:22.669158 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:22.669427 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.669975 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.670610 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:22.670784 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:22.670825 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:22.673107 (+ 2282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:56817: BlockingRecv error: recv got EOF from 127.31.15.195:56817 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:23.434993 14278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:23.430123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:40687)
0809 16:35:23.430280 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:23.430293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:23.430370 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:23.430547 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:23.430549 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:23.430720 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:23.430769 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.430770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.431397 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.431397 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.431999 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.432002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.432393 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.432394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.432458 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.432901 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:23.432906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:23.434884 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
W20260809 16:35:23.435025 14279 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:23.430202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:40687 (local address 127.31.15.254:45071)
0809 16:35:23.430306 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:23.430308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:23.430349 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:35:23.430421 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:23.430421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:23.430432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:23.430488 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:23.430833 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.431345 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.432078 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.432307 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.432366 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.434970 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:40687: BlockingRecv error: recv got EOF from 127.31.15.193:40687 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:23.564163 14280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:23.559590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:45483)
0809 16:35:23.559744 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:23.559757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:23.559827 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:23.559957 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:23.559958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:23.560154 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:23.560200 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.560202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.560859 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.560861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.561389 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.561391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.561731 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.561732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.561787 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.562184 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:23.562189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:23.564103 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
W20260809 16:35:23.564249 14279 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:23.559667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:45483 (local address 127.31.15.254:45071)
0809 16:35:23.559747 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:23.559748 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:23.559816 (+ 68us) server_negotiation.cc:407] Connection header received
0809 16:35:23.559879 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:23.559880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:23.559895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:23.559948 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:23.560291 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.560788 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.561469 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.561665 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.561722 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.564193 (+ 2471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:45483: BlockingRecv error: recv got EOF from 127.31.15.194:45483 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:23 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:23.677861 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:23.673612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:53611)
0809 16:35:23.673767 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:23.673778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:23.673838 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:23.673951 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:23.673952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:23.674115 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:23.674165 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.674167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.674748 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.674749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.675290 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:23.675293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.675577 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.675577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:23.675613 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.676014 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:23.676019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:23.677797 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
W20260809 16:35:23.677938 14279 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:23.673725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:53611 (local address 127.31.15.254:45071)
0809 16:35:23.673774 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:23.673776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:23.673823 (+ 47us) server_negotiation.cc:407] Connection header received
0809 16:35:23.673888 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:23.673889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:23.673900 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:23.673936 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:23.674215 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.674697 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.675344 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:23.675527 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:23.675575 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:23.677885 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:53611: BlockingRecv error: recv got EOF from 127.31.15.195:53611 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:24.439993 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:24.435470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:40417)
0809 16:35:24.435639 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:24.435651 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:24.435711 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:24.435840 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:24.435841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:24.436024 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:24.436072 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.436074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.436733 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.436735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.437447 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.437450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.437757 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.437758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.437793 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.438194 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:24.438198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:24.439890 (+ 1692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
W20260809 16:35:24.440048 14284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:24.435522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:40417 (local address 127.31.15.254:45071)
0809 16:35:24.435643 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:24.435645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:24.435694 (+ 49us) server_negotiation.cc:407] Connection header received
0809 16:35:24.435756 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:24.435757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:24.435767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:24.435826 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:24.436129 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.436630 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.437509 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.437705 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.437755 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.439985 (+ 2230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:40417: BlockingRecv error: recv got EOF from 127.31.15.193:40417 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:24.569067 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:24.564681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:37335)
0809 16:35:24.564821 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:24.564834 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:24.564904 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:24.565051 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:24.565052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:24.565273 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:24.565334 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.565337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.565967 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.565968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.566500 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.566502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.566778 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.566778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.566814 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.567202 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:24.567206 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:24.568992 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
W20260809 16:35:24.569159 14284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:24.564769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:37335 (local address 127.31.15.254:45071)
0809 16:35:24.564816 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:24.564818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:24.564897 (+ 79us) server_negotiation.cc:407] Connection header received
0809 16:35:24.564964 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:24.564965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:24.564981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:24.565037 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:24.565408 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.565908 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.566558 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.566734 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.566776 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.569089 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:37335: BlockingRecv error: recv got EOF from 127.31.15.194:37335 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:24 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:24.682874 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:24.678376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:50151)
0809 16:35:24.678526 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:24.678537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:24.678599 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:24.678719 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:24.678720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:24.678890 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:24.678939 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.678940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.679619 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.679620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.680175 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:24.680177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.680486 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.680487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:24.680537 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.680935 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:24.680939 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:24.682799 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
W20260809 16:35:24.682960 14284 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:24.678468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:50151 (local address 127.31.15.254:45071)
0809 16:35:24.678544 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:24.678546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:24.678597 (+ 51us) server_negotiation.cc:407] Connection header received
0809 16:35:24.678641 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:24.678642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:24.678656 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:24.678708 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:24.679007 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.679549 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.680237 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:24.680421 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:24.680470 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:24.682890 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:50151: BlockingRecv error: recv got EOF from 127.31.15.195:50151 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:25.444746 14289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:25.440457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:55747)
0809 16:35:25.440616 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:25.440629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:25.440687 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:25.440804 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:25.440805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:25.440977 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:25.441024 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.441026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.441659 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.441659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.442195 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.442197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.442492 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.442493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.442544 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.442957 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:25.442960 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:25.444673 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
W20260809 16:35:25.444875 14291 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:25.440520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:55747 (local address 127.31.15.254:45071)
0809 16:35:25.440661 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:25.440663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:25.440671 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:25.440742 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:25.440742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:25.440752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:25.440799 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:25.441076 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.441599 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.442251 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.442432 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.442473 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.444800 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:55747: BlockingRecv error: recv got EOF from 127.31.15.193:55747 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:25.573648 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:25.569518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:49313)
0809 16:35:25.569648 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:25.569660 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:25.569717 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:25.569828 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:25.569828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:25.570004 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:25.570054 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.570055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.570660 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.570661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.571181 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.571182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.571497 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.571498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.571538 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.571939 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:25.571943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:25.573587 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
W20260809 16:35:25.573733 14291 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:25.569574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:49313 (local address 127.31.15.254:45071)
0809 16:35:25.569621 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:25.569622 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:25.569703 (+ 81us) server_negotiation.cc:407] Connection header received
0809 16:35:25.569757 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:25.569758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:25.569768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:25.569814 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:25.570109 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.570603 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.571234 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.571444 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.571490 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.573674 (+ 2184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:49313: BlockingRecv error: recv got EOF from 127.31.15.194:49313 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:25 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:25.687731 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:25.683342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:38507)
0809 16:35:25.683480 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:25.683490 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:25.683548 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:25.683702 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:25.683703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:25.683877 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:25.683924 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.683926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.684542 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.684543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.685062 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:25.685064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.685367 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.685368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:25.685457 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.685891 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:25.685895 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:25.687653 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
W20260809 16:35:25.687808 14291 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:25.683404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:38507 (local address 127.31.15.254:45071)
0809 16:35:25.683459 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:25.683461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:25.683541 (+ 80us) server_negotiation.cc:407] Connection header received
0809 16:35:25.683588 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:25.683588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:25.683600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:25.683652 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:25.683981 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.684468 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.685118 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:25.685291 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:25.685339 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:25.687754 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:38507: BlockingRecv error: recv got EOF from 127.31.15.195:38507 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:26.449776 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:26.445213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:39117)
0809 16:35:26.445338 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:26.445348 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:26.445407 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:26.445551 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:26.445552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:26.445722 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:26.445768 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.445770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.446440 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.446440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.447100 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.447104 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.447479 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.447480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.447523 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.447918 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:26.447922 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:26.449718 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
W20260809 16:35:26.449891 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:26.445280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39117 (local address 127.31.15.254:45071)
0809 16:35:26.445412 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:26.445414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:26.445455 (+ 41us) server_negotiation.cc:407] Connection header received
0809 16:35:26.445476 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:26.445476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:26.445488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:26.445538 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:26.445858 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.446372 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.447167 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.447425 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.447476 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.449814 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:39117: BlockingRecv error: recv got EOF from 127.31.15.193:39117 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:26.578621 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:26.574104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:43915)
0809 16:35:26.574251 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:26.574264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:26.574325 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:26.574457 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:26.574458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:26.574647 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:26.574694 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.574695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.575374 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.575375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.575927 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.575929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.576425 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.576427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.576494 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.576939 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:26.576943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:26.578553 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
W20260809 16:35:26.578735 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:26.574230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:43915 (local address 127.31.15.254:45071)
0809 16:35:26.574292 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:26.574294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:26.574306 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:35:26.574388 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:26.574388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:26.574401 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:26.574441 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:26.574771 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.575265 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.576038 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.576338 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.576426 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.578671 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:43915: BlockingRecv error: recv got EOF from 127.31.15.194:43915 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:26 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:26.692438 14299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:26.688203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:56371)
0809 16:35:26.688332 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:26.688344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:26.688405 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:26.688526 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:26.688527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:26.688702 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:26.688749 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.688751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.689361 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.689362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.689909 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:26.689910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.690192 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.690193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:26.690230 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.690682 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:26.690686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:26.692369 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
W20260809 16:35:26.692543 14296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:26.688256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56371 (local address 127.31.15.254:45071)
0809 16:35:26.688336 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:26.688338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:26.688401 (+ 63us) server_negotiation.cc:407] Connection header received
0809 16:35:26.688456 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:26.688457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:26.688469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:26.688521 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:26.688801 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.689304 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.689965 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:26.690144 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:26.690189 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:26.692456 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:56371: BlockingRecv error: recv got EOF from 127.31.15.195:56371 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:27.454370 14300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:27.450220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:42531)
0809 16:35:27.450347 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:27.450358 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:27.450407 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:27.450530 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:27.450531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:27.450711 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:27.450757 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.450759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.451390 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.451391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.451928 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.451930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.452239 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.452240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.452276 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.452709 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:27.452713 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:27.454298 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
W20260809 16:35:27.454463 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:27.450294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:42531 (local address 127.31.15.254:45071)
0809 16:35:27.450415 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:27.450417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:27.450425 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:27.450448 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:27.450448 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:27.450459 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:27.450521 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:27.450814 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.451338 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.451998 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.452189 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.452237 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.454391 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:42531: BlockingRecv error: recv got EOF from 127.31.15.193:42531 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":57,"threads_started":1}
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:27.588543 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:27.579074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:35675)
0809 16:35:27.579206 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:27.579216 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:27.579262 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:27.579411 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:27.579412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:27.579578 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:27.579626 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.579628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.580220 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.580220 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.580729 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.580730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.581047 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.581048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.581083 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.586723 (+ 5640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:27.586728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:27.588479 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
W20260809 16:35:27.588646 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:27.579177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:35675 (local address 127.31.15.254:45071)
0809 16:35:27.579241 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:27.579242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:27.579253 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:35:27.579335 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:27.579335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:27.579346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:27.579400 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:27.579677 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.580167 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.580791 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.580986 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.581047 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.588568 (+ 7521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:35675: BlockingRecv error: recv got EOF from 127.31.15.194:35675 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:27 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.195@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:27.697288 14303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:27.692846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.195:37335)
0809 16:35:27.692969 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:27.692980 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:27.693044 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:27.693176 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:27.693177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:27.693353 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:27.693400 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.693402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.694039 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.694040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.694572 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:27.694574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.694932 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.694932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:27.694990 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.695472 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:27.695477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:27.697227 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260809 16:35:27.697392 14301 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:27.692900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:37335 (local address 127.31.15.254:45071)
0809 16:35:27.692964 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:27.692966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:27.693033 (+ 67us) server_negotiation.cc:407] Connection header received
0809 16:35:27.693102 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:27.693102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:27.693117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:27.693161 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:27.693468 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.693964 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.694667 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:27.694865 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:27.694928 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:27.697326 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.195:37335: BlockingRecv error: recv got EOF from 127.31.15.195:37335 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 16:35:28 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:28 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.193@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:28.459352 14304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:28.454901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.193:56069)
0809 16:35:28.455064 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:28.455079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:28.455133 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:28.455255 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:28.455255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:28.455450 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:28.455498 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:28.455500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.456114 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.456114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:28.456650 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:28.456652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.456945 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.456945 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:28.456984 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:28.457465 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:28.457470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:28.459249 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
W20260809 16:35:28.459450 14305 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:28.455026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:56069 (local address 127.31.15.254:45071)
0809 16:35:28.455142 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:28.455144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:28.455151 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:35:28.455175 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:28.455175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:28.455187 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:28.455244 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:28.455559 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.456061 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.456709 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.456893 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.456937 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:28.459376 (+ 2439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.193:56069: BlockingRecv error: recv got EOF from 127.31.15.193:56069 (error 108)
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":51,"threads_started":1}
Aug 09 16:35:28 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 16:35:28 dist-test-slave-21b9 krb5kdc[12629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.31.15.194@KRBTEST.COM for oryx/127.31.15.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 16:35:28.593678 14306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:28.589054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45071 (local address 127.31.15.194:51611)
0809 16:35:28.589203 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:28.589214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:28.589278 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:28.589415 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:28.589416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:28.589589 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:28.589639 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:28.589640 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.590323 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.590324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:28.591054 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:28.591058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.591413 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.591414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:28.591453 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:28.591876 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:28.591880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:28.593606 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.31.15.254:45071: Server oryx/127.31.15.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
W20260809 16:35:28.593747 14305 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:28.589150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:51611 (local address 127.31.15.254:45071)
0809 16:35:28.589210 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:28.589212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:28.589269 (+ 57us) server_negotiation.cc:407] Connection header received
0809 16:35:28.589337 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:28.589338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:28.589350 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:28.589403 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:28.589690 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.590223 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.591126 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.591348 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.591407 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:28.593694 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.31.15.194:51611: BlockingRecv error: recv got EOF from 127.31.15.194:51611 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:35:28.622295 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13534
I20260809 16:35:28.636768 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13776
I20260809 16:35:28.645004 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13998
I20260809 16:35:28.654785 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 13421
2026-08-09T16:35:28Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17572 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T16:35:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:28Z Disabled control of system clock
I20260809 16:35:28.686725 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39805
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:39805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 16:35:28.766346 14315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:28.766554 14315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:28.766582 14315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:28.767752 14315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:28.767794 14315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:28.767812 14315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:28.767827 14315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:28.767841 14315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:28.769181 14315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:39805
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:28.769424 14315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:28.769690 14315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:28.774130 14326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:28.774169 14323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:28.774307 14315 server_base.cc:1034] running on GCE node
W20260809 16:35:28.774189 14322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:28.774587 14315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:28.774827 14315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:28.775972 14315 hybrid_clock.cc:648] HybridClock initialized: now 1786293328775952 us; error 29 us; skew 500 ppm
I20260809 16:35:28.777105 14315 webserver.cc:460] Webserver started at http://127.31.15.254:39049/ using document root <none> and password file <none>
I20260809 16:35:28.777324 14315 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:28.777367 14315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:28.777490 14315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:28.778465 14315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "acb7a292bfe7422fa5e0d187cf064908"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "0d984d1b39ab0ae32b6f80d1086a9a91"
server_key_iv: "328558ef64cd0f4432d5eda1c18cf85d"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.778780 14315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "acb7a292bfe7422fa5e0d187cf064908"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "0d984d1b39ab0ae32b6f80d1086a9a91"
server_key_iv: "328558ef64cd0f4432d5eda1c18cf85d"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.780019 14315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:28.780748 14333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:28.780933 14315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:28.781014 14315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "acb7a292bfe7422fa5e0d187cf064908"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "0d984d1b39ab0ae32b6f80d1086a9a91"
server_key_iv: "328558ef64cd0f4432d5eda1c18cf85d"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.781075 14315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:28.802980 14315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:28.803328 14315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:28.803465 14315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:28.807308 14315 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:39805
I20260809 16:35:28.807387 14417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:39805 every 8 connection(s)
I20260809 16:35:28.807684 14315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:28.808416 14418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:28.810887 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 14315
I20260809 16:35:28.810971 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:28.811147 31807 external_mini_cluster.cc:1428] Setting key 27b26731138120c90145aafb2240b0bb
I20260809 16:35:28.811156 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908: Bootstrap starting.
I20260809 16:35:28.811944 14418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:28.812292 14418 log.cc:824] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:28.813088 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908: No bootstrap required, opened a new log
I20260809 16:35:28.814868 14418 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } }
I20260809 16:35:28.815035 14418 raft_consensus.cc:374] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:28.815085 14418 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb7a292bfe7422fa5e0d187cf064908, State: Initialized, Role: FOLLOWER
I20260809 16:35:28.815209 14418 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } }
I20260809 16:35:28.815565 14418 raft_consensus.cc:388] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:28.815620 14418 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:28.815675 14418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:28.816367 14418 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } }
I20260809 16:35:28.816493 14418 leader_election.cc:302] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acb7a292bfe7422fa5e0d187cf064908; no voters:
I20260809 16:35:28.816660 14418 leader_election.cc:288] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:28.816704 14428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:28.816877 14428 raft_consensus.cc:686] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [term 1 LEADER]: Becoming Leader. State: Replica: acb7a292bfe7422fa5e0d187cf064908, State: Running, Role: LEADER
I20260809 16:35:28.816948 14418 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:28.817004 14428 consensus_queue.cc:237] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } }
I20260809 16:35:28.817152 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:28.811654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47430 (local address 127.31.15.254:39805)
0809 16:35:28.811850 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:28.811859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:28.811907 (+ 48us) server_negotiation.cc:407] Connection header received
0809 16:35:28.812453 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:28.812462 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:28.812546 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:28.812682 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:28.812909 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.814250 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.814982 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:28.815614 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:28.815752 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:28.816391 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:28.816396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:28.816396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:28.816612 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:28.816657 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:28.816673 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:28.816745 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:28.816865 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 16:35:28.816951 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260809 16:35:28.817605 14431 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acb7a292bfe7422fa5e0d187cf064908. Latest consensus state: current_term: 1 leader_uuid: "acb7a292bfe7422fa5e0d187cf064908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } } }
I20260809 16:35:28.817807 14429 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acb7a292bfe7422fa5e0d187cf064908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acb7a292bfe7422fa5e0d187cf064908" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39805 } } }
I20260809 16:35:28.817902 14431 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:28.817960 14429 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:28.818233 14447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:28.818714 14447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:28.820286 14447 catalog_manager.cc:1347] Generated new cluster ID: 998f0e961f0148a1ac76976e85e6d133
I20260809 16:35:28.820339 14447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:28.838531 14447 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:28.839037 14447 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:28.841728 14447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acb7a292bfe7422fa5e0d187cf064908: Generated new TSK 0
I20260809 16:35:28.841871 14447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:35:28.850484 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--builtin_ntp_servers=127.31.15.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 16:35:28.969643 14458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:28.969944 14458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:28.969985 14458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:28.971643 14458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:28.971705 14458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:28.971743 14458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:28.973637 14458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:28.973888 14458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:28.974226 14458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:28.979410 14472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:28.979447 14475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:28.979626 14458 server_base.cc:1034] running on GCE node
W20260809 16:35:28.979455 14471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:28.979909 14458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:28.980146 14458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:28.981320 14458 hybrid_clock.cc:648] HybridClock initialized: now 1786293328981301 us; error 36 us; skew 500 ppm
I20260809 16:35:28.982654 14458 webserver.cc:460] Webserver started at http://127.31.15.193:32769/ using document root <none> and password file <none>
I20260809 16:35:28.982892 14458 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:28.982954 14458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:28.983083 14458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:28.984220 14458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "4b7c3ebb3cf5404eb6d06409bbd22093"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "4052a052506d7e691912214f0f1e2502"
server_key_iv: "1c48e2f4d54d03bba426bfe3d33f9b0b"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.984570 14458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "4b7c3ebb3cf5404eb6d06409bbd22093"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "4052a052506d7e691912214f0f1e2502"
server_key_iv: "1c48e2f4d54d03bba426bfe3d33f9b0b"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.985899 14458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:35:28.986722 14482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:28.986891 14458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:35:28.986981 14458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "4b7c3ebb3cf5404eb6d06409bbd22093"
format_stamp: "Formatted at 2026-08-09 16:35:28 on dist-test-slave-21b9"
server_key: "4052a052506d7e691912214f0f1e2502"
server_key_iv: "1c48e2f4d54d03bba426bfe3d33f9b0b"
server_key_version: "encryptionkey@0"
I20260809 16:35:28.987048 14458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:28.998768 14458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:28.999051 14458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:28.999171 14458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:28.999404 14458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:28.999711 14458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:28.999744 14458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:28.999773 14458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:28.999795 14458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.005158 14458 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:38551
I20260809 16:35:29.005232 14658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:38551 every 8 connection(s)
I20260809 16:35:29.005502 14458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:35:29.006700 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 14458
I20260809 16:35:29.006793 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:29.006999 31807 external_mini_cluster.cc:1428] Setting key 6a788a787a47544333380b6525340f28
I20260809 16:35:29.007972 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--builtin_ntp_servers=127.31.15.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 16:35:29.011370 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.006202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:40037 (local address 127.31.15.254:39805)
0809 16:35:29.006246 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.006248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.006454 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:35:29.007146 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.007148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.007161 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.007242 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.008077 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.008631 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.009887 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.010135 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.010192 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.011042 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.011043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.011044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.011048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.011064 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.011068 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.011112 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.011241 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.011289 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:35:29.011574 14665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.006174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39805 (local address 127.31.15.193:40037)
0809 16:35:29.006355 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.006393 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.007004 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.007334 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.007339 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.007349 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.007983 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.007991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.008718 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.008720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.009808 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.009810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.010205 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.010206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.010356 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.010912 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.010913 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.010941 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.010964 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.011162 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.011168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.011171 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.011230 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.011358 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260809 16:35:29.012022 14663 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39805
I20260809 16:35:29.012111 14663 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.012290 14663 heartbeater.cc:511] Master 127.31.15.254:39805 requested a full tablet report, sending...
I20260809 16:35:29.012846 14359 ts_manager.cc:194] Registered new tserver with Master: 4b7c3ebb3cf5404eb6d06409bbd22093 (127.31.15.193:38551)
I20260809 16:35:29.013764 14359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.193:40037
W20260809 16:35:29.088814 14666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.089013 14666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.089032 14666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.090186 14666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.090225 14666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.090248 14666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:29.091591 14666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.091764 14666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.092010 14666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.096566 14679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.096607 14682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.096593 14666 server_base.cc:1034] running on GCE node
W20260809 16:35:29.096568 14674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.096938 14666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.097157 14666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.098291 14666 hybrid_clock.cc:648] HybridClock initialized: now 1786293329098282 us; error 30 us; skew 500 ppm
I20260809 16:35:29.099406 14666 webserver.cc:460] Webserver started at http://127.31.15.194:37165/ using document root <none> and password file <none>
I20260809 16:35:29.099635 14666 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.099689 14666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.099814 14666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.100770 14666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "6839d75d646a4317875b3deaea4efaae"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "feed9c6b0d89880b9f1ac2136510722f"
server_key_iv: "54d791a97693a2e8583f2e9966765bf0"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.101097 14666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "6839d75d646a4317875b3deaea4efaae"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "feed9c6b0d89880b9f1ac2136510722f"
server_key_iv: "54d791a97693a2e8583f2e9966765bf0"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.102295 14666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:35:29.103003 14690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.103157 14666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:29.103250 14666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "6839d75d646a4317875b3deaea4efaae"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "feed9c6b0d89880b9f1ac2136510722f"
server_key_iv: "54d791a97693a2e8583f2e9966765bf0"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.103340 14666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.123821 14666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.124068 14666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.124189 14666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.124377 14666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:29.124666 14666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:29.124699 14666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.124725 14666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:29.124742 14666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.130062 14666 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:37979
I20260809 16:35:29.130118 14879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:37979 every 8 connection(s)
I20260809 16:35:29.130388 14666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:35:29.133585 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 14666
I20260809 16:35:29.133666 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:29.133834 31807 external_mini_cluster.cc:1428] Setting key d4c7b64127a3a221b530e8394f3a5805
I20260809 16:35:29.134859 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--builtin_ntp_servers=127.31.15.212:45525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 16:35:29.135360 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.131032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:36157 (local address 127.31.15.254:39805)
0809 16:35:29.131091 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.131093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.131308 (+ 215us) server_negotiation.cc:407] Connection header received
0809 16:35:29.131766 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.131767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.131776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.131830 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.132399 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.132881 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.133849 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.134103 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.134173 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.135098 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.135099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.135100 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.135106 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.135122 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.135126 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.135181 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.135236 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.135260 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:35:29.135533 14882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.131013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39805 (local address 127.31.15.194:36157)
0809 16:35:29.131186 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.131222 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.131690 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.131877 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.131880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.131888 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.132339 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.132343 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.132938 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.132938 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.133769 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.133771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.134214 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.134215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.134364 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.134896 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.134896 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.134931 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.134957 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.135185 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.135189 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.135191 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.135235 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.135344 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260809 16:35:29.136018 14880 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39805
I20260809 16:35:29.136111 14880 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.136287 14880 heartbeater.cc:511] Master 127.31.15.254:39805 requested a full tablet report, sending...
I20260809 16:35:29.136672 14359 ts_manager.cc:194] Registered new tserver with Master: 6839d75d646a4317875b3deaea4efaae (127.31.15.194:37979)
I20260809 16:35:29.137137 14359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.194:36157
W20260809 16:35:29.214625 14883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.214812 14883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.214831 14883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.215945 14883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.215984 14883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.216006 14883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:29.217306 14883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.217505 14883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.217736 14883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.222476 14889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.222512 14890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.222520 14892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.222591 14883 server_base.cc:1034] running on GCE node
I20260809 16:35:29.222888 14883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.223114 14883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.224247 14883 hybrid_clock.cc:648] HybridClock initialized: now 1786293329224223 us; error 41 us; skew 500 ppm
I20260809 16:35:29.225440 14883 webserver.cc:460] Webserver started at http://127.31.15.195:36503/ using document root <none> and password file <none>
I20260809 16:35:29.225651 14883 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.225694 14883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.225792 14883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.226765 14883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "5921b5303cb840889c82f71300955dca"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "46e95ec8ed3bbb31f7376b3499e7847e"
server_key_iv: "a4614e4005b7119969c658b0044c9b32"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.227070 14883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "5921b5303cb840889c82f71300955dca"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "46e95ec8ed3bbb31f7376b3499e7847e"
server_key_iv: "a4614e4005b7119969c658b0044c9b32"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.228305 14883 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:29.228925 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.229054 14883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:29.229146 14883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "5921b5303cb840889c82f71300955dca"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "46e95ec8ed3bbb31f7376b3499e7847e"
server_key_iv: "a4614e4005b7119969c658b0044c9b32"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.229203 14883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.237041 14883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.237269 14883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.237386 14883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.237571 14883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:29.237847 14883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:29.237879 14883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.237906 14883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:29.237931 14883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.243105 14883 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:41635
I20260809 16:35:29.243165 15078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:41635 every 8 connection(s)
I20260809 16:35:29.243446 14883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:29.247928 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.244091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:34517 (local address 127.31.15.254:39805)
0809 16:35:29.244147 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.244149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.244361 (+ 212us) server_negotiation.cc:407] Connection header received
0809 16:35:29.244808 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.244810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.244820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.244869 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.245451 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.245933 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.246826 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.247004 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.247045 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.247735 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.247736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.247737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.247741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.247753 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.247756 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.247793 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.247844 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.247864 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:35:29.248095 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.244066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39805 (local address 127.31.15.195:34517)
0809 16:35:29.244252 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.244284 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.244733 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.244917 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.244920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.244928 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.245391 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.245395 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.245993 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.245994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.246765 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:29.246767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:29.247057 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:29.247057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:29.247153 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:29.247621 (+ 468us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.247621 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.247648 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.247669 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.247794 (+ 125us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.247797 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.247799 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.247841 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.247910 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260809 16:35:29.248538 15081 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39805
I20260809 16:35:29.248616 15081 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.248755 15081 heartbeater.cc:511] Master 127.31.15.254:39805 requested a full tablet report, sending...
I20260809 16:35:29.249071 14359 ts_manager.cc:194] Registered new tserver with Master: 5921b5303cb840889c82f71300955dca (127.31.15.195:41635)
I20260809 16:35:29.249416 14359 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.195:34517
I20260809 16:35:29.250145 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 14883
I20260809 16:35:29.250231 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:29.250378 31807 external_mini_cluster.cc:1428] Setting key 6cc374e2c711911bdd1d411eb3cdae54
I20260809 16:35:29.251111 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:35:29.252794 14424 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:29.252356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47432 (local address 127.31.15.254:39805)
0809 16:35:29.252410 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.252411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.252507 (+ 96us) server_negotiation.cc:407] Connection header received
0809 16:35:29.252567 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.252568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.252577 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.252618 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.252736 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47432: BlockingRecv error: recv got EOF from 127.0.0.1:47432 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:35:29.252876 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 14458
I20260809 16:35:29.277973 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 14666
I20260809 16:35:29.296983 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 14883
I20260809 16:35:29.305966 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 14315
2026-08-09T16:35:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (648 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T16:35:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:29Z Disabled control of system clock
I20260809 16:35:29.335215 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39971
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:39971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:35:29.414198 15095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.414386 15095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.414407 15095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.415472 15095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:29.415511 15095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.415522 15095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.415534 15095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:29.415544 15095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:29.416803 15095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:39971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.416994 15095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.417224 15095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.422029 15106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.422010 15110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.422200 15095 server_base.cc:1034] running on GCE node
W20260809 16:35:29.422205 15114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.422542 15095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.422801 15095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.423961 15095 hybrid_clock.cc:648] HybridClock initialized: now 1786293329423944 us; error 32 us; skew 500 ppm
I20260809 16:35:29.425732 15095 webserver.cc:460] Webserver started at http://127.31.15.254:32901/ using document root <none> and password file <none>
I20260809 16:35:29.425942 15095 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.425992 15095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.426118 15095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.427459 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "c1e5f834d12f4fa6b1e0c5bad482386d"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "c06c85477e01abc096cca7e02f5a3927"
server_key_iv: "6075fd2b59f0aeefe738dde84232e72b"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.427822 15095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "c1e5f834d12f4fa6b1e0c5bad482386d"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "c06c85477e01abc096cca7e02f5a3927"
server_key_iv: "6075fd2b59f0aeefe738dde84232e72b"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.429065 15095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:29.429829 15121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.429982 15095 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:29.430065 15095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "c1e5f834d12f4fa6b1e0c5bad482386d"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "c06c85477e01abc096cca7e02f5a3927"
server_key_iv: "6075fd2b59f0aeefe738dde84232e72b"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.430123 15095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.433645 15095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.433876 15095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.434000 15095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.437677 15095 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:39971
I20260809 16:35:29.437714 15199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:39971 every 8 connection(s)
I20260809 16:35:29.438002 15095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:29.438864 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:29.439100 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 15095
I20260809 16:35:29.439188 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:29.439384 31807 external_mini_cluster.cc:1428] Setting key ea46af6d542b81eabce68dca0570130d
I20260809 16:35:29.441746 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: Bootstrap starting.
I20260809 16:35:29.441793 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.439855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.31.15.254:39971)
0809 16:35:29.440074 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.440083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.440114 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:35:29.440626 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.440636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.440686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.440831 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.441375 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.441378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.441378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.441383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.441428 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.441442 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.441520 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.441569 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.441623 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260809 16:35:29.442428 15200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:29.442669 15200 log.cc:824] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:29.443238 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: No bootstrap required, opened a new log
I20260809 16:35:29.444564 15200 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } }
I20260809 16:35:29.444697 15200 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:29.444741 15200 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e5f834d12f4fa6b1e0c5bad482386d, State: Initialized, Role: FOLLOWER
I20260809 16:35:29.444886 15200 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } }
I20260809 16:35:29.444976 15200 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:29.445003 15200 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:29.445035 15200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:29.445833 15200 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } }
I20260809 16:35:29.445919 15200 leader_election.cc:302] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1e5f834d12f4fa6b1e0c5bad482386d; no voters:
I20260809 16:35:29.446058 15200 leader_election.cc:288] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:29.446112 15206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:29.446286 15206 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [term 1 LEADER]: Becoming Leader. State: Replica: c1e5f834d12f4fa6b1e0c5bad482386d, State: Running, Role: LEADER
I20260809 16:35:29.446290 15200 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:29.446451 15206 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } }
I20260809 16:35:29.446792 15212 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e5f834d12f4fa6b1e0c5bad482386d. Latest consensus state: current_term: 1 leader_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } } }
I20260809 16:35:29.446883 15212 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:29.447052 15211 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1e5f834d12f4fa6b1e0c5bad482386d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39971 } } }
I20260809 16:35:29.447134 15211 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d [sys.catalog]: This master's current role is: LEADER
W20260809 16:35:29.448011 15232 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:35:29.448055 15232 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:35:29.448115 15222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:29.448282 15222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:29.449673 15222 catalog_manager.cc:1347] Generated new cluster ID: 449a1e0dde1f4134a4b2ad898baac658
I20260809 16:35:29.449726 15222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:29.464838 15222 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:29.464999 15222 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:29.471776 15222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e5f834d12f4fa6b1e0c5bad482386d: Generated new TSK 0
I20260809 16:35:29.471936 15222 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:35:29.479097 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--builtin_ntp_servers=127.31.15.212:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:35:29.562532 15235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.562776 15235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.562813 15235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.564044 15235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.564092 15235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.564131 15235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:29.565500 15235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.565737 15235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.566015 15235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.571610 15244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.571617 15242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.571728 15235 server_base.cc:1034] running on GCE node
W20260809 16:35:29.571625 15246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.572057 15235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.572288 15235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.573462 15235 hybrid_clock.cc:648] HybridClock initialized: now 1786293329573453 us; error 30 us; skew 500 ppm
I20260809 16:35:29.574622 15235 webserver.cc:460] Webserver started at http://127.31.15.193:33741/ using document root <none> and password file <none>
I20260809 16:35:29.574837 15235 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.574885 15235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.575017 15235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.576126 15235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "23c70f77769e4d11b9847afb6c9cb927"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "92a9b78ed9d0628d97cc631c0eb9878e"
server_key_iv: "e930592b58e1c49ef6210db79caa6c74"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.576467 15235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "23c70f77769e4d11b9847afb6c9cb927"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "92a9b78ed9d0628d97cc631c0eb9878e"
server_key_iv: "e930592b58e1c49ef6210db79caa6c74"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.577735 15235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:35:29.578423 15256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.578588 15235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:29.578675 15235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "23c70f77769e4d11b9847afb6c9cb927"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "92a9b78ed9d0628d97cc631c0eb9878e"
server_key_iv: "e930592b58e1c49ef6210db79caa6c74"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.578739 15235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.582218 15235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.582530 15235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.582666 15235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.582890 15235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:29.583206 15235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:29.583257 15235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.583319 15235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:29.583349 15235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.589844 15235 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:33175
I20260809 16:35:29.589905 15422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:33175 every 8 connection(s)
I20260809 16:35:29.590201 15235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:35:29.593010 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.590922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:52731 (local address 127.31.15.254:39971)
0809 16:35:29.590987 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.590990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.591225 (+ 235us) server_negotiation.cc:407] Connection header received
0809 16:35:29.591749 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.591750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.591756 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.591807 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.592833 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.592834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.592835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.592838 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.592851 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.592854 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.592896 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.592945 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.592956 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:35:29.593174 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.590890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39971 (local address 127.31.15.193:52731)
0809 16:35:29.591098 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.591139 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.591659 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.591859 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.591862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.591870 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.592711 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.592711 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.592738 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.592764 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.592895 (+ 131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.592897 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.592899 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.592947 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.592995 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260809 16:35:29.593657 15423 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39971
I20260809 16:35:29.593744 15423 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.593887 15423 heartbeater.cc:511] Master 127.31.15.254:39971 requested a full tablet report, sending...
I20260809 16:35:29.594352 15142 ts_manager.cc:194] Registered new tserver with Master: 23c70f77769e4d11b9847afb6c9cb927 (127.31.15.193:33175)
I20260809 16:35:29.594939 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 15235
I20260809 16:35:29.595043 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:29.595233 31807 external_mini_cluster.cc:1428] Setting key b8839da4f3fa48a7bde649362493ada4
I20260809 16:35:29.596060 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--builtin_ntp_servers=127.31.15.212:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:35:29.677078 15428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.677265 15428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.677285 15428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.678406 15428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.678445 15428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.678470 15428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:29.679808 15428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.680027 15428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.680292 15428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.684890 15438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.684885 15441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.684890 15436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.685251 15428 server_base.cc:1034] running on GCE node
I20260809 16:35:29.685422 15428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.685611 15428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.686748 15428 hybrid_clock.cc:648] HybridClock initialized: now 1786293329686737 us; error 25 us; skew 500 ppm
I20260809 16:35:29.687978 15428 webserver.cc:460] Webserver started at http://127.31.15.194:38619/ using document root <none> and password file <none>
I20260809 16:35:29.688205 15428 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.688251 15428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.688367 15428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.689385 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "e412aff3df2548878310ae675981f879"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b741da0d0ffd77425c00c6480619713b"
server_key_iv: "362eccec1ccf952626de58e02ac595ea"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.689698 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "e412aff3df2548878310ae675981f879"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b741da0d0ffd77425c00c6480619713b"
server_key_iv: "362eccec1ccf952626de58e02ac595ea"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.691004 15428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:29.691727 15452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.691874 15428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.691951 15428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "e412aff3df2548878310ae675981f879"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b741da0d0ffd77425c00c6480619713b"
server_key_iv: "362eccec1ccf952626de58e02ac595ea"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.692024 15428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.695137 15428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.695389 15428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.695503 15428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.695691 15428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:29.695981 15428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:29.696012 15428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.696040 15428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:29.696063 15428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.701293 15428 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:45247
I20260809 16:35:29.701359 15632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:45247 every 8 connection(s)
I20260809 16:35:29.701610 15428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:35:29.704373 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.702298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:59529 (local address 127.31.15.254:39971)
0809 16:35:29.702356 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.702357 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.702517 (+ 160us) server_negotiation.cc:407] Connection header received
0809 16:35:29.702911 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.702913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.702919 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.702984 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.704055 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.704057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.704057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.704062 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.704079 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.704084 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.704145 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.704293 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.704309 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:35:29.704484 15638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.702232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39971 (local address 127.31.15.194:59529)
0809 16:35:29.702405 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.702441 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.702836 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.703035 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.703038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.703046 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.703883 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.703883 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.703909 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.703935 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.704180 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.704187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.704189 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.704264 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.704314 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 16:35:29.704890 15634 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39971
I20260809 16:35:29.704969 15634 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.705127 15634 heartbeater.cc:511] Master 127.31.15.254:39971 requested a full tablet report, sending...
I20260809 16:35:29.705478 15142 ts_manager.cc:194] Registered new tserver with Master: e412aff3df2548878310ae675981f879 (127.31.15.194:45247)
I20260809 16:35:29.711078 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 15428
I20260809 16:35:29.711170 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:29.711373 31807 external_mini_cluster.cc:1428] Setting key 9d6bf02725d75d68762aec622c335b11
I20260809 16:35:29.712196 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--builtin_ntp_servers=127.31.15.212:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 16:35:29.792289 15640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:29.792472 15640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:29.792492 15640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:29.793606 15640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:29.793644 15640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:29.793668 15640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:29.795018 15640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:39971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:29.795231 15640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:29.795524 15640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:29.800102 15650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:29.800127 15653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.800143 15640 server_base.cc:1034] running on GCE node
W20260809 16:35:29.800127 15649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:29.800460 15640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:29.800705 15640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:29.801836 15640 hybrid_clock.cc:648] HybridClock initialized: now 1786293329801819 us; error 38 us; skew 500 ppm
I20260809 16:35:29.802883 15640 webserver.cc:460] Webserver started at http://127.31.15.195:35737/ using document root <none> and password file <none>
I20260809 16:35:29.803088 15640 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:29.803131 15640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:29.803254 15640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:29.804257 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "259380811d30445a9d2ad6bcc651f6dc"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b9a079850f5f5ae26fab9fe97e095be5"
server_key_iv: "1c22deaedb9e2c20beec1e6ba1c49a83"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.804584 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "259380811d30445a9d2ad6bcc651f6dc"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b9a079850f5f5ae26fab9fe97e095be5"
server_key_iv: "1c22deaedb9e2c20beec1e6ba1c49a83"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.805770 15640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:35:29.806458 15662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.806640 15640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:29.806733 15640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "259380811d30445a9d2ad6bcc651f6dc"
format_stamp: "Formatted at 2026-08-09 16:35:29 on dist-test-slave-21b9"
server_key: "b9a079850f5f5ae26fab9fe97e095be5"
server_key_iv: "1c22deaedb9e2c20beec1e6ba1c49a83"
server_key_version: "encryptionkey@0"
I20260809 16:35:29.806797 15640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:29.810113 15640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:29.810323 15640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:29.810422 15640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:29.810585 15640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:29.810835 15640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:29.810868 15640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.810906 15640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:29.810925 15640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:29.816787 15640 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:35561
I20260809 16:35:29.816826 15835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:35561 every 8 connection(s)
I20260809 16:35:29.817152 15640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:29.819826 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.817806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:55473 (local address 127.31.15.254:39971)
0809 16:35:29.817849 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.817851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.818049 (+ 198us) server_negotiation.cc:407] Connection header received
0809 16:35:29.818462 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.818464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.818470 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.818528 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.819630 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:29.819631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:29.819631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:35:29.819636 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:29.819651 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:35:29.819656 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:29.819706 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:29.819762 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 16:35:29.819773 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:35:29.819991 15841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:29.817781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:39971 (local address 127.31.15.195:55473)
0809 16:35:29.817937 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:29.817976 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:29.818370 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:29.818600 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:29.818605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:29.818614 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:29.819478 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:35:29.819478 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:29.819504 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:35:29.819531 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:29.819710 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:29.819712 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:29.819715 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:29.819759 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:35:29.819803 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260809 16:35:29.820436 15836 heartbeater.cc:347] Connected to a master server at 127.31.15.254:39971
I20260809 16:35:29.820518 15836 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:29.820653 15836 heartbeater.cc:511] Master 127.31.15.254:39971 requested a full tablet report, sending...
I20260809 16:35:29.820982 15142 ts_manager.cc:194] Registered new tserver with Master: 259380811d30445a9d2ad6bcc651f6dc (127.31.15.195:35561)
I20260809 16:35:29.827235 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 15640
I20260809 16:35:29.827358 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:29.827543 31807 external_mini_cluster.cc:1428] Setting key 938a53af257570c84581b5c3542371cf
I20260809 16:35:29.828326 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 16:35:29.830024 15203 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:35:29.829549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42704 (local address 127.31.15.254:39971)
0809 16:35:29.829602 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:29.829604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:29.829719 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:35:29.829784 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:29.829785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:29.829790 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:29.829836 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:29.829951 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42704: BlockingRecv error: recv got EOF from 127.0.0.1:42704 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:35:29.830054 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 15235
I20260809 16:35:29.851438 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 15428
I20260809 16:35:29.862389 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 15640
I20260809 16:35:29.868575 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 15095
2026-08-09T16:35:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (564 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:35:29 dist-test-slave-21b9 krb5kdc[15855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:35:29 dist-test-slave-21b9 krb5kdc[15855](info): set up 2 sockets
Aug 09 16:35:29 dist-test-slave-21b9 krb5kdc[15855](info): commencing operation
krb5kdc: starting...
W20260809 16:35:31.022265 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:35:31.137830 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:33925
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:33925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:31.217844 15876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:31.218043 15876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:31.218065 15876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:31.219136 15876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:35:31.219167 15876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:31.219180 15876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:31.219192 15876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:35:31.219202 15876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:35:31.220664 15876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:33925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:31.220855 15876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:31.221024 15876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:31.225620 15888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.225611 15889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.225600 15891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.226542 15876 server_base.cc:1034] running on GCE node
I20260809 16:35:31.226744 15876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:31.226985 15876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:31.228163 15876 hybrid_clock.cc:648] HybridClock initialized: now 1786293331228144 us; error 31 us; skew 500 ppm
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:31.230623 15876 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:35:31.231035 15876 webserver.cc:460] Webserver started at http://127.31.15.254:41747/ using document root <none> and password file <none>
I20260809 16:35:31.231230 15876 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:31.231297 15876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:31.231431 15876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:31.232501 15876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "d8d394d830d04b568d63091ec4ade906"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "c2cf04f7cd2aa77deec2bb65bcef4041"
server_key_iv: "65108d572f307797a086688e843a7c46"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.232826 15876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "d8d394d830d04b568d63091ec4ade906"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "c2cf04f7cd2aa77deec2bb65bcef4041"
server_key_iv: "65108d572f307797a086688e843a7c46"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.234076 15876 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:31.234689 15901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.234848 15876 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:31.234932 15876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "d8d394d830d04b568d63091ec4ade906"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "c2cf04f7cd2aa77deec2bb65bcef4041"
server_key_iv: "65108d572f307797a086688e843a7c46"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.234998 15876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:31.252859 15876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:31.254601 15876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:31.254736 15876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:31.258651 15876 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:33925
I20260809 16:35:31.258684 15974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:33925 every 8 connection(s)
I20260809 16:35:31.259022 15876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:35:31.260015 15976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:31.262241 15976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: Bootstrap starting.
I20260809 16:35:31.262354 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 15876
I20260809 16:35:31.262456 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:35:31.262635 31807 external_mini_cluster.cc:1428] Setting key e8e52edde7008d57c4e8914f96c56a6b
I20260809 16:35:31.262842 15976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:31.263162 15976 log.cc:824] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:31.263850 15976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: No bootstrap required, opened a new log
I20260809 16:35:31.265609 15976 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } }
I20260809 16:35:31.265749 15976 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:31.265781 15976 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d394d830d04b568d63091ec4ade906, State: Initialized, Role: FOLLOWER
I20260809 16:35:31.265901 15976 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } }
I20260809 16:35:31.265982 15976 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:35:31.266016 15976 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:35:31.266084 15976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:31.266634 15976 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } }
I20260809 16:35:31.266733 15976 leader_election.cc:302] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8d394d830d04b568d63091ec4ade906; no voters:
I20260809 16:35:31.266888 15976 leader_election.cc:288] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:35:31.266909 15984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:31.267036 15984 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [term 1 LEADER]: Becoming Leader. State: Replica: d8d394d830d04b568d63091ec4ade906, State: Running, Role: LEADER
I20260809 16:35:31.267122 15984 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } }
I20260809 16:35:31.267452 15976 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:35:31.267485 15988 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8d394d830d04b568d63091ec4ade906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } } }
I20260809 16:35:31.267505 15986 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8d394d830d04b568d63091ec4ade906. Latest consensus state: current_term: 1 leader_uuid: "d8d394d830d04b568d63091ec4ade906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8d394d830d04b568d63091ec4ade906" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33925 } } }
I20260809 16:35:31.267671 15986 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [sys.catalog]: This master's current role is: LEADER
I20260809 16:35:31.267657 15988 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906 [sys.catalog]: This master's current role is: LEADER
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
W20260809 16:35:31.269311 16006 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:35:31.269363 16006 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:35:31.269464 15990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:35:31.269691 15990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:35:31.271117 15990 catalog_manager.cc:1347] Generated new cluster ID: f723a46bf2e940948f118f63434f04c5
I20260809 16:35:31.271160 15990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:35:31.273178 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.263089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.31.15.254:33925)
0809 16:35:31.263300 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.263310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.263337 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:35:31.263743 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.263749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.263799 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.263924 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:31.264523 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.265381 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.266235 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.266694 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.266795 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.269562 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:31.269566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:31.269566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:31.269577 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:31.271603 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.271971 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.271982 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.271982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.272088 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.272285 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.272285 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.272286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.272582 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:31.272743 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.272819 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.272953 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260809 16:35:31.275077 15990 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:35:31.275595 15990 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:35:31.283432 15990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: Generated new TSK 0
I20260809 16:35:31.283627 15990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:35:31.310633 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:31.390875 16016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:31.391111 16016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:31.391137 16016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:31.392315 16016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:31.392356 16016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:31.392381 16016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:35:31.393791 16016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:31.394028 16016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:31.394232 16016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:31.398905 16027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.398869 16030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.398877 16023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.399168 16016 server_base.cc:1034] running on GCE node
I20260809 16:35:31.399418 16016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:31.399617 16016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:31.400770 16016 hybrid_clock.cc:648] HybridClock initialized: now 1786293331400753 us; error 34 us; skew 500 ppm
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:31.403203 16016 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:35:31.403652 16016 webserver.cc:460] Webserver started at http://127.31.15.193:42619/ using document root <none> and password file <none>
I20260809 16:35:31.403868 16016 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:31.403916 16016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:31.404044 16016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:31.405023 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "f24a76682bca4346afcc50a2018c7d73"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "078003c6b182d57d0371ab64abb9af96"
server_key_iv: "d3f411d7cf3973efa4755e54572580e2"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.405347 16016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "f24a76682bca4346afcc50a2018c7d73"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "078003c6b182d57d0371ab64abb9af96"
server_key_iv: "d3f411d7cf3973efa4755e54572580e2"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.406569 16016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:35:31.407199 16041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.407366 16016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:31.407447 16016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "f24a76682bca4346afcc50a2018c7d73"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "078003c6b182d57d0371ab64abb9af96"
server_key_iv: "d3f411d7cf3973efa4755e54572580e2"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.407519 16016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:31.417577 16016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:31.419227 16016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:31.419394 16016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:31.419584 16016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:31.419855 16016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:31.419888 16016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.419915 16016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:31.419932 16016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.425523 16016 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:33975
I20260809 16:35:31.425563 16204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:33975 every 8 connection(s)
I20260809 16:35:31.425848 16016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:31.433871 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.426519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39805 (local address 127.31.15.254:33925)
0809 16:35:31.426582 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.426584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.426755 (+ 171us) server_negotiation.cc:407] Connection header received
0809 16:35:31.427195 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.427196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.427205 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.427254 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:31.428173 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.428678 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.429593 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.429779 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.429823 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.431762 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:31.431763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:31.431763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:31.431768 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:31.432856 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.433190 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.433190 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.433191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.433233 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.433399 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.433399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.433399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.433488 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:31.433546 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.433742 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.433806 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:35:31.434020 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.426488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:33925 (local address 127.31.15.193:39805)
0809 16:35:31.426643 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.426680 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.427122 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.427359 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.427363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.427917 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:31.428098 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.428103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.428743 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.428744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.429533 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.429535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.429831 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.429831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.429923 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.430341 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:31.430349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:31.431682 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:31.432924 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.432931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.432932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.433132 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.433273 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.433273 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.433274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.433357 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.433542 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:31.433545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:31.433649 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.433725 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.433835 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 16:35:31.434510 16205 heartbeater.cc:347] Connected to a master server at 127.31.15.254:33925
I20260809 16:35:31.434602 16205 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:31.434765 16205 heartbeater.cc:511] Master 127.31.15.254:33925 requested a full tablet report, sending...
I20260809 16:35:31.435247 15924 ts_manager.cc:194] Registered new tserver with Master: f24a76682bca4346afcc50a2018c7d73 (127.31.15.193:33975)
I20260809 16:35:31.435482 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 16016
I20260809 16:35:31.435591 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:35:31.435774 31807 external_mini_cluster.cc:1428] Setting key 2daa29ec9ba8ff57295b814e819385bc
I20260809 16:35:31.436084 15924 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:39805
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:35:31.459475 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:31.541038 16211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:31.541239 16211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:31.541270 16211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:31.542374 16211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:31.542415 16211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:31.542448 16211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:35:31.543936 16211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:31.544174 16211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:31.544384 16211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:31.549168 16219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.549176 16216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.549288 16211 server_base.cc:1034] running on GCE node
W20260809 16:35:31.549176 16217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.549629 16211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:31.549882 16211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:31.551023 16211 hybrid_clock.cc:648] HybridClock initialized: now 1786293331550986 us; error 53 us; skew 500 ppm
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:31.553510 16211 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:35:31.553928 16211 webserver.cc:460] Webserver started at http://127.31.15.194:38685/ using document root <none> and password file <none>
I20260809 16:35:31.554157 16211 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:31.554206 16211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:31.554329 16211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:31.555382 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "fd1f889017424791904981e6c39b576a"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "b64d8bc025d33fdb85d6ec25c006c587"
server_key_iv: "9fa67bfefef0ef7a6a3f81b3c9b646b4"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.555697 16211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "fd1f889017424791904981e6c39b576a"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "b64d8bc025d33fdb85d6ec25c006c587"
server_key_iv: "9fa67bfefef0ef7a6a3f81b3c9b646b4"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.556876 16211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:35:31.557559 16229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.557714 16211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:35:31.557798 16211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "fd1f889017424791904981e6c39b576a"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "b64d8bc025d33fdb85d6ec25c006c587"
server_key_iv: "9fa67bfefef0ef7a6a3f81b3c9b646b4"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.557857 16211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:31.585151 16211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:31.586985 16211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:31.587126 16211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:31.587371 16211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:31.587688 16211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:31.587723 16211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.587754 16211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:31.587770 16211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.593576 16211 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:32951
I20260809 16:35:31.593655 16400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:32951 every 8 connection(s)
I20260809 16:35:31.593997 16211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:35:31.594230 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 16211
I20260809 16:35:31.594336 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:35:31.594554 31807 external_mini_cluster.cc:1428] Setting key 9c67a1ea0ff915f1affcc60fea2cefad
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
I20260809 16:35:31.603799 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.595047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:56447 (local address 127.31.15.254:33925)
0809 16:35:31.595097 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.595099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.595109 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:35:31.595680 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.595681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.595692 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.595745 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:31.596660 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.597325 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.598643 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.598819 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.598862 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.601276 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:31.601277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:31.601278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:31.601284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:31.602499 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.602923 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.602923 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.602924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.602968 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.603201 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.603201 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.603201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.603329 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:31.603378 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.603676 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.603734 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:35:31.604045 16404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.594771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:33925 (local address 127.31.15.194:56447)
0809 16:35:31.594921 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.594975 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.595574 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.595822 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.595825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.596380 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:31.596576 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.596581 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.597416 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.597417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.598573 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.598576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.598888 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.598889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.599032 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.599606 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:31.599616 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:31.601188 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:31.602577 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.602584 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.602586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.602854 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.603016 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.603017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.603017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.603144 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.603375 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:31.603381 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:31.603552 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.603662 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.603816 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260809 16:35:31.604494 16402 heartbeater.cc:347] Connected to a master server at 127.31.15.254:33925
I20260809 16:35:31.604574 16402 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:31.604776 16402 heartbeater.cc:511] Master 127.31.15.254:33925 requested a full tablet report, sending...
I20260809 16:35:31.605139 15922 ts_manager.cc:194] Registered new tserver with Master: fd1f889017424791904981e6c39b576a (127.31.15.194:32951)
I20260809 16:35:31.605520 15922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:56447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:35:31.620050 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:35:31.700209 16408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:35:31.700398 16408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:35:31.700419 16408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:35:31.701555 16408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:35:31.701597 16408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:35:31.701624 16408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:35:31.703043 16408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:35:31.703286 16408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:35:31.703522 16408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:35:31.708069 16415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:35:31.708124 16416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.708124 16408 server_base.cc:1034] running on GCE node
W20260809 16:35:31.708076 16418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:35:31.708509 16408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:35:31.708767 16408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:35:31.709903 16408 hybrid_clock.cc:648] HybridClock initialized: now 1786293331709894 us; error 29 us; skew 500 ppm
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:35:31.712414 16408 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:35:31.712805 16408 webserver.cc:460] Webserver started at http://127.31.15.195:45483/ using document root <none> and password file <none>
I20260809 16:35:31.713011 16408 fs_manager.cc:359] Metadata directory not provided
I20260809 16:35:31.713064 16408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:35:31.713187 16408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:35:31.714154 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "62cbdcbbfb6647bfb74f0838dc079b8c"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "987686c2375af8d361416b206c7a86a9"
server_key_iv: "6e101f2453e93c3d16bd9ab6b6616f4f"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.714455 16408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "62cbdcbbfb6647bfb74f0838dc079b8c"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "987686c2375af8d361416b206c7a86a9"
server_key_iv: "6e101f2453e93c3d16bd9ab6b6616f4f"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.715685 16408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:35:31.716320 16429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.716461 16408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:35:31.716542 16408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "62cbdcbbfb6647bfb74f0838dc079b8c"
format_stamp: "Formatted at 2026-08-09 16:35:31 on dist-test-slave-21b9"
server_key: "987686c2375af8d361416b206c7a86a9"
server_key_iv: "6e101f2453e93c3d16bd9ab6b6616f4f"
server_key_version: "encryptionkey@0"
I20260809 16:35:31.716599 16408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:35:31.743927 16408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:35:31.745980 16408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:35:31.746115 16408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:35:31.746320 16408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:35:31.746604 16408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:35:31.746637 16408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.746665 16408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:35:31.746688 16408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:35:31.752032 16408 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:44681
I20260809 16:35:31.752089 16597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:44681 every 8 connection(s)
I20260809 16:35:31.752382 16408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:35:31.754554 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 16408
I20260809 16:35:31.754669 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:35:31.754853 31807 external_mini_cluster.cc:1428] Setting key b25cace81d70d2f94b6b410a4650ac83
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[15855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293331, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:35:31.761091 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.753055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:47709 (local address 127.31.15.254:33925)
0809 16:35:31.753122 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.753124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.753270 (+ 146us) server_negotiation.cc:407] Connection header received
0809 16:35:31.753773 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.753775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.753787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.753841 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:31.754904 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.755464 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.756480 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.756677 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.756727 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.758681 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:31.758683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:31.758683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:31.758689 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:31.759875 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.760195 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.760196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.760196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.760244 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.760504 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.760504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.760505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.760613 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:31.760669 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.760917 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.760993 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:35:31.761265 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.752999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:33925 (local address 127.31.15.195:47709)
0809 16:35:31.753162 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.753197 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.753673 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.753917 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.753920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.754530 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:35:31.754809 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.754816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.755546 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.755547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.756410 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.756413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.756739 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.756740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.756845 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.757231 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:35:31.757238 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:35:31.758597 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:35:31.759942 (+ 1345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.759947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.759948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.760141 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.760337 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:35:31.760338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:35:31.760339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:35:31.760444 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:35:31.760670 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:35:31.760676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:35:31.760803 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.760889 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.761040 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260809 16:35:31.761739 16598 heartbeater.cc:347] Connected to a master server at 127.31.15.254:33925
I20260809 16:35:31.761829 16598 heartbeater.cc:464] Registering TS with master...
I20260809 16:35:31.762006 16598 heartbeater.cc:511] Master 127.31.15.254:33925 requested a full tablet report, sending...
I20260809 16:35:31.762333 15922 ts_manager.cc:194] Registered new tserver with Master: 62cbdcbbfb6647bfb74f0838dc079b8c (127.31.15.195:44681)
I20260809 16:35:31.762671 15922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:47709
I20260809 16:35:31.766496 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:35:31.773500 15982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.767831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.31.15.254:33925)
0809 16:35:31.767975 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.767977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.767986 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:35:31.768052 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.768053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.768062 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.768106 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:35:31.768661 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.769231 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.769934 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.770135 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.770186 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.771212 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:35:31.771213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:35:31.771213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:35:31.771220 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:35:31.772465 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.772762 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.772763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.772764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.772815 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:35:31.772993 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:35:31.772993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:35:31.772994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:35:31.773133 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:35:31.773219 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.773343 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.773419 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:35:31.774896 15922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:35:31.776451 15922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:35:31.781764 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:32951 (local address 127.0.0.1:57758)
0809 16:35:31.778445 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.778483 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.778549 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.778814 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.778817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.778820 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:31.778882 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.778883 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.779939 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.779940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.780809 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.780810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.781488 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.781489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.781565 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.781579 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.781641 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.781694 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":49,"threads_started":1}
I20260809 16:35:31.781996 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.31.15.194:32951)
0809 16:35:31.778635 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.778640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.778648 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:31.778687 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.778689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.778705 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.778784 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:31.778967 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.779856 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.780909 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.781409 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.781481 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.781520 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.781873 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.781916 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260809 16:35:31.782788 16616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:44681 (local address 127.0.0.1:59494)
0809 16:35:31.778926 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.778936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.778989 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.779367 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.779368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.779370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:31.779414 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.779416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.780664 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.780665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.781568 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.781570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.782536 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.782538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.782605 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.782609 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.782671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.782714 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":51,"threads_started":1}
I20260809 16:35:31.782802 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.31.15.195:44681)
0809 16:35:31.778743 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.778746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.778985 (+ 239us) server_negotiation.cc:407] Connection header received
0809 16:35:31.779056 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.779058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.779073 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.779293 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:31.779475 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.780559 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.781650 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.782448 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.782521 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.782575 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.782684 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.782722 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 16:35:31.782768 16298 tablet_service.cc:1467] Processing CreateTablet for tablet ba1444fd57df48629e5384b2263a344c (DEFAULT_TABLE table=test-table [id=d59a506c9ea8412aae448461d46c597f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:31.783123 16298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1444fd57df48629e5384b2263a344c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:31.783852 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.31.15.193:33975)
0809 16:35:31.778923 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.778926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.779121 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:35:31.779164 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.779166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.779182 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.779249 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:31.779479 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.780565 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.782700 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.783400 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.783463 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.783514 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.783719 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.783767 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260809 16:35:31.783880 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.778491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33975 (local address 127.0.0.1:44824)
0809 16:35:31.779050 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.779056 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.779090 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.779352 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.779353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.779355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:31.779402 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.779404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.781658 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.781658 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.782608 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.782610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.783564 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.783565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.783623 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.783628 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.783683 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.783719 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":39,"threads_started":1}
I20260809 16:35:31.783785 16503 tablet_service.cc:1467] Processing CreateTablet for tablet ba1444fd57df48629e5384b2263a344c (DEFAULT_TABLE table=test-table [id=d59a506c9ea8412aae448461d46c597f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:31.784121 16503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1444fd57df48629e5384b2263a344c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:31.785977 16626 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Bootstrap starting.
I20260809 16:35:31.786706 16626 tablet_bootstrap.cc:654] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:31.786746 16627 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Bootstrap starting.
I20260809 16:35:31.787314 16627 tablet_bootstrap.cc:654] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:31.787384 16626 log.cc:824] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:31.787516 16100 tablet_service.cc:1467] Processing CreateTablet for tablet ba1444fd57df48629e5384b2263a344c (DEFAULT_TABLE table=test-table [id=d59a506c9ea8412aae448461d46c597f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:35:31.787829 16100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba1444fd57df48629e5384b2263a344c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:35:31.787897 16627 log.cc:824] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:31.788203 16626 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: No bootstrap required, opened a new log
I20260809 16:35:31.788266 16626 ts_tablet_manager.cc:1397] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:35:31.788473 16627 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: No bootstrap required, opened a new log
I20260809 16:35:31.788515 16627 ts_tablet_manager.cc:1397] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:35:31.789929 16627 raft_consensus.cc:348] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.790045 16627 raft_consensus.cc:374] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:31.790063 16627 raft_consensus.cc:729] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62cbdcbbfb6647bfb74f0838dc079b8c, State: Initialized, Role: FOLLOWER
I20260809 16:35:31.790148 16627 consensus_queue.cc:260] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.790385 16627 ts_tablet_manager.cc:1428] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:35:31.790410 16598 heartbeater.cc:503] Master 127.31.15.254:33925 was elected leader, sending a full tablet report...
I20260809 16:35:31.790270 16626 raft_consensus.cc:348] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.790728 16626 raft_consensus.cc:374] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:31.790778 16626 raft_consensus.cc:729] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd1f889017424791904981e6c39b576a, State: Initialized, Role: FOLLOWER
I20260809 16:35:31.790906 16626 consensus_queue.cc:260] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.791159 16626 ts_tablet_manager.cc:1428] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:35:31.791177 16633 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: Bootstrap starting.
I20260809 16:35:31.791301 16402 heartbeater.cc:503] Master 127.31.15.254:33925 was elected leader, sending a full tablet report...
I20260809 16:35:31.791814 16633 tablet_bootstrap.cc:654] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: Neither blocks nor log segments found. Creating new log.
I20260809 16:35:31.792316 16633 log.cc:824] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: Log is configured to *not* fsync() on all Append() calls
I20260809 16:35:31.792927 16633 tablet_bootstrap.cc:492] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: No bootstrap required, opened a new log
I20260809 16:35:31.792982 16633 ts_tablet_manager.cc:1397] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:35:31.794420 16633 raft_consensus.cc:348] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.794535 16633 raft_consensus.cc:374] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:35:31.794565 16633 raft_consensus.cc:729] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24a76682bca4346afcc50a2018c7d73, State: Initialized, Role: FOLLOWER
I20260809 16:35:31.794667 16633 consensus_queue.cc:260] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.794883 16633 ts_tablet_manager.cc:1428] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:35:31.794960 16205 heartbeater.cc:503] Master 127.31.15.254:33925 was elected leader, sending a full tablet report...
I20260809 16:35:31.821566 16631 raft_consensus.cc:482] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:35:31.821692 16631 raft_consensus.cc:504] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.822041 16631 leader_election.cc:288] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd1f889017424791904981e6c39b576a (127.31.15.194:32951), f24a76682bca4346afcc50a2018c7d73 (127.31.15.193:33975)
I20260809 16:35:31.824847 16615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.822245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:40715 (local address 127.31.15.194:32951)
0809 16:35:31.822310 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.822312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.822324 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:35:31.822352 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.822352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.822365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.822428 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:31.822578 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.823226 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824150 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.824533 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824595 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.824604 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.824742 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.824779 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:35:31.824859 16625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.822338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:54683 (local address 127.31.15.193:33975)
0809 16:35:31.822388 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.822389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.822507 (+ 118us) server_negotiation.cc:407] Connection header received
0809 16:35:31.822540 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.822541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.822550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.822601 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:35:31.822716 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.823317 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824184 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.824556 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824622 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.824630 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.824763 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.824796 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:35:31.824865 16600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.822177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:32951 (local address 127.31.15.195:40715)
0809 16:35:31.822236 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.822248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.822293 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.822437 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.822437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.822439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:31.822504 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.822506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.823311 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.823312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.824067 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.824070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.824607 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.824682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.824685 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.824724 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.824761 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:35:31.824874 16641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.822267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33975 (local address 127.31.15.195:54683)
0809 16:35:31.822439 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.822449 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:35:31.822503 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:35:31.822607 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:35:31.822607 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:35:31.822609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:35:31.822662 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.822663 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.823381 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.823382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.824101 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:35:31.824103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.824632 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.824633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:35:31.824698 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.824701 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:35:31.824750 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:35:31.824782 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260809 16:35:31.825316 16326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1444fd57df48629e5384b2263a344c" candidate_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1f889017424791904981e6c39b576a" is_pre_election: true
I20260809 16:35:31.825317 16133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1444fd57df48629e5384b2263a344c" candidate_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24a76682bca4346afcc50a2018c7d73" is_pre_election: true
I20260809 16:35:31.825474 16133 raft_consensus.cc:2454] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cbdcbbfb6647bfb74f0838dc079b8c in term 0.
I20260809 16:35:31.825474 16326 raft_consensus.cc:2454] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62cbdcbbfb6647bfb74f0838dc079b8c in term 0.
I20260809 16:35:31.825667 16433 leader_election.cc:302] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cbdcbbfb6647bfb74f0838dc079b8c, fd1f889017424791904981e6c39b576a; no voters:
I20260809 16:35:31.825771 16631 raft_consensus.cc:2781] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:35:31.825822 16631 raft_consensus.cc:482] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:35:31.825851 16631 raft_consensus.cc:3037] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:31.826507 16631 raft_consensus.cc:504] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.826606 16631 leader_election.cc:288] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [CANDIDATE]: Term 1 election: Requested vote from peers fd1f889017424791904981e6c39b576a (127.31.15.194:32951), f24a76682bca4346afcc50a2018c7d73 (127.31.15.193:33975)
I20260809 16:35:31.826771 16326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1444fd57df48629e5384b2263a344c" candidate_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd1f889017424791904981e6c39b576a"
I20260809 16:35:31.826804 16133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba1444fd57df48629e5384b2263a344c" candidate_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24a76682bca4346afcc50a2018c7d73"
I20260809 16:35:31.826851 16326 raft_consensus.cc:3037] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:31.826864 16133 raft_consensus.cc:3037] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:35:31.827741 16133 raft_consensus.cc:2454] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cbdcbbfb6647bfb74f0838dc079b8c in term 1.
I20260809 16:35:31.827741 16326 raft_consensus.cc:2454] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62cbdcbbfb6647bfb74f0838dc079b8c in term 1.
I20260809 16:35:31.827888 16433 leader_election.cc:302] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62cbdcbbfb6647bfb74f0838dc079b8c, f24a76682bca4346afcc50a2018c7d73; no voters:
I20260809 16:35:31.827966 16631 raft_consensus.cc:2781] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:35:31.828109 16631 raft_consensus.cc:686] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 1 LEADER]: Becoming Leader. State: Replica: 62cbdcbbfb6647bfb74f0838dc079b8c, State: Running, Role: LEADER
I20260809 16:35:31.828194 16631 consensus_queue.cc:237] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } }
I20260809 16:35:31.828951 15922 catalog_manager.cc:5495] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 62cbdcbbfb6647bfb74f0838dc079b8c (127.31.15.195). New cstate: current_term: 1 leader_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62cbdcbbfb6647bfb74f0838dc079b8c" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 44681 } health_report { overall_health: HEALTHY } } }
I20260809 16:35:31.838395 16620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.836473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.31.15.195:44681)
0809 16:35:31.836511 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:35:31.836513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:35:31.836521 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:35:31.836572 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:35:31.836573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:35:31.836594 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:35:31.836647 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:35:31.836729 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.837145 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.837794 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:35:31.837964 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:35:31.838005 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:35:31.838101 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:35:31.838185 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:35:31.838237 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:35:31.838294 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:35:31.838308 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:35:31.839900 16133 raft_consensus.cc:1261] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73 [term 1 FOLLOWER]: Refusing update from remote peer 62cbdcbbfb6647bfb74f0838dc079b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:31.839900 16326 raft_consensus.cc:1261] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 1 FOLLOWER]: Refusing update from remote peer 62cbdcbbfb6647bfb74f0838dc079b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:35:31.840216 16631 consensus_queue.cc:1035] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd1f889017424791904981e6c39b576a" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 32951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:31.840342 16642 consensus_queue.cc:1035] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24a76682bca4346afcc50a2018c7d73" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:35:31.843201 16647 mvcc.cc:205] Tried to move back new op lower bound from 7316657487214170112 to 7316657487168933888. Current Snapshot: MvccSnapshot[applied={T|T < 7316657487214170112}]
I20260809 16:35:31.843201 16648 mvcc.cc:205] Tried to move back new op lower bound from 7316657487214170112 to 7316657487168933888. Current Snapshot: MvccSnapshot[applied={T|T < 7316657487214170112}]
I20260809 16:35:31.844578 15924 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:31.844664 15924 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:35:31.845340 15924 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8d394d830d04b568d63091ec4ade906: Sending DeleteTablet for 3 replicas of tablet ba1444fd57df48629e5384b2263a344c
I20260809 16:35:31.845615 16298 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1444fd57df48629e5384b2263a344c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:31 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:57758
I20260809 16:35:31.845651 16100 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1444fd57df48629e5384b2263a344c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:31 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:44824
I20260809 16:35:31.845656 16503 tablet_service.cc:1514] Processing DeleteTablet for tablet ba1444fd57df48629e5384b2263a344c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:35:31 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:59494
I20260809 16:35:31.845888 16662 tablet_replica.cc:330] T ba1444fd57df48629e5384b2263a344c P f24a76682bca4346afcc50a2018c7d73: stopping tablet replica
I20260809 16:35:31.845924 16664 tablet_replica.cc:330] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: stopping tablet replica
I20260809 16:35:31.846012 16663 tablet_replica.cc:330] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: stopping tablet replica
I20260809 16:35:31.846050 16664 raft_consensus.cc:2229] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:35:31.846117 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 16016
I20260809 16:35:31.846131 16663 raft_consensus.cc:2229] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:35:31.846221 16663 raft_consensus.cc:2258] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:31.846210 16664 raft_consensus.cc:2258] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:35:31.849241 16663 ts_tablet_manager.cc:1905] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:31.850973 16663 ts_tablet_manager.cc:1918] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:31.851039 16663 log.cc:1196] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/ba1444fd57df48629e5384b2263a344c
I20260809 16:35:31.851250 16663 ts_tablet_manager.cc:1939] T ba1444fd57df48629e5384b2263a344c P fd1f889017424791904981e6c39b576a: Deleting consensus metadata
I20260809 16:35:31.852665 15904 catalog_manager.cc:4841] TS fd1f889017424791904981e6c39b576a (127.31.15.194:32951): tablet ba1444fd57df48629e5384b2263a344c (table test-table [id=d59a506c9ea8412aae448461d46c597f]) successfully deleted
I20260809 16:35:31.869997 16664 ts_tablet_manager.cc:1905] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:35:31.871563 16664 ts_tablet_manager.cc:1918] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:35:31.871673 16664 log.cc:1196] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/ba1444fd57df48629e5384b2263a344c
I20260809 16:35:31.871953 16664 ts_tablet_manager.cc:1939] T ba1444fd57df48629e5384b2263a344c P 62cbdcbbfb6647bfb74f0838dc079b8c: Deleting consensus metadata
I20260809 16:35:31.872417 15902 catalog_manager.cc:4841] TS 62cbdcbbfb6647bfb74f0838dc079b8c (127.31.15.195:44681): tablet ba1444fd57df48629e5384b2263a344c (table test-table [id=d59a506c9ea8412aae448461d46c597f]) successfully deleted
W20260809 16:35:31.873976 15904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:33975 (error 108)
I20260809 16:35:31.874226 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 16211
I20260809 16:35:31.876746 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:35:31.874304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:33975 (local address 127.0.0.1:44826)
0809 16:35:31.876666 (+ 2362us) negotiation.cc:107] Waiting for socket to connect
0809 16:35:31.876712 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2339}
W20260809 16:35:31.879032 15904 catalog_manager.cc:4584] TS f24a76682bca4346afcc50a2018c7d73 (127.31.15.193:33975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba1444fd57df48629e5384b2263a344c: Network error: Client connection negotiation failed: client connection to 127.31.15.193:33975: connect: Connection refused (error 111)
I20260809 16:35:31.891472 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 16408
I20260809 16:35:31.913625 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 15876
2026-08-09T16:35:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2049 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[16669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[16669](info): set up 2 sockets
Aug 09 16:35:31 dist-test-slave-21b9 krb5kdc[16669](info): commencing operation
krb5kdc: starting...
W20260809 16:35:33.064019 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:35:33 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:35:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:35:33Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangeradmin_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangeradmin_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangerlookup_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangerlookup_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
I20260809 16:35:33.217520 31807 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfvbJG8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/postgres -l logfile start
2026-08-09 16:35:35.505 UTC [16723] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:35:35.505 UTC [16723] LOG: listening on IPv4 address "127.31.15.212", port 39471
2026-08-09 16:35:35.509 UTC [16723] LOG: listening on Unix socket "/tmp/.s.PGSQL.39471"
2026-08-09 16:35:35.530 UTC [16729] LOG: database system was shut down at 2026-08-09 16:35:34 UTC
2026-08-09 16:35:35.533 UTC [16723] LOG: database system is ready to accept connections
I20260809 16:35:36.608489 31807 mini_postgres.cc:96] Postgres bound to 39471
2026-08-09 16:35:36.616 UTC [16738] FATAL: database "slave" does not exist
127.31.15.212:39471 - accepting connections
I20260809 16:35:36.616930 31807 mini_ranger.cc:161] Starting Ranger...
I20260809 16:35:36.631816 31807 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:35:37.040385 31807 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:35:37.040474 31807 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin
2026-08-09 16:35:37,343 [I] DB FLAVOR :POSTGRES
2026-08-09 16:35:37,343 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:35:37,343 [I] Checking connection..
2026-08-09 16:35:37,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:35:37,602 [I] Checking connection passed.
2026-08-09 16:35:37,602 [I] --------- Verifying version history table ---------
2026-08-09 16:35:37,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:35:37,851 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:35:37,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:35:38,098 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:35:38,098 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:35:38,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:35:38.325 UTC [16834] WARNING: there is no transaction in progress
2026-08-09 16:35:38,330 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:35:38,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:35:38,564 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:35:38,564 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:35:38,565 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:35:38,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:35:39,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-21b9.c.gcp-upstream.internal','N') ;"
2026-08-09 16:35:39,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:35:39,542 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:35:39,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:35:39,789 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:35:39,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:35:40,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:35:40,285 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:35:40,285 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:35:41.165 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.180 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.195 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.210 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.224 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.241 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.291 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.297 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.310 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.319 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.328 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.340 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.349 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.355 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.361 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41.367 UTC [17017] WARNING: there is no transaction in progress
2026-08-09 16:35:41,928 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:35:41,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:35:42,170 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:35:42,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:35:42,426 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:35:42,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:35:42,682 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:35:42,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:35:42,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:35:43,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-21b9.c.gcp-upstream.internal';"
2026-08-09 16:35:43,418 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:35:43,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:35:43,679 [I] DB_PATCHES have already been applied
I20260809 16:35:43.685369 31807 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfvbJG8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:35:43.685432 31807 mini_ranger.cc:193] Using host: 127.31.15.212
I20260809 16:35:43.690047 31807 mini_ranger.cc:227] Ranger admin URL: http://127.31.15.212:35447
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:35:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:35:44.965766 31807 mini_ranger.cc:160] Time spent starting Ranger: real 8.349s user 0.001s sys 0.010s
Aug 09, 2026 4:36:04 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:36:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:36:04 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:36:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:36:05 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:36:06.073508 31807 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangerkms_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/rangerkms_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.15.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.15.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:36:06 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:36:06.155427 31807 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:36:06.166077 31807 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:36:06.918186 31807 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:36:06.918288 31807 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms
I20260809 16:36:06.918308 31807 mini_ranger_kms.cc:227] Using postgres at 127.31.15.212:39471
2026-08-09 16:36:07,180 [I] DB FLAVOR :POSTGRES
2026-08-09 16:36:07,180 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:36:07,180 [I] Checking connection
2026-08-09 16:36:07,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:36:07,484 [I] connection success
2026-08-09 16:36:07,485 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:36:07,485 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:36:07,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:36:07,719 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:36:07,720 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:36:07,720 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:36:07,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:36:07,974 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:36:08.399107 31807 mini_ranger_kms.cc:310] Created kms service
I20260809 16:36:08.603448 31807 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:36:08.644079 31807 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:36:08 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293366, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:09.012516 31807 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:36:09.012635 31807 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:36:09.012656 31807 mini_ranger_kms.cc:240] Using host: 127.31.15.212
I20260809 16:36:09.017964 31807 mini_ranger_kms.cc:276] Ranger KMS PID: 17293
I20260809 16:36:09.018074 31807 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.15.212:36871
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:36:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 16:36:11.595891 31807 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.440s user 0.004s sys 0.238s
I20260809 16:36:11.595999 31807 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:36:11.596079 31807 mini_ranger_kms.cc:401] 127.31.15.212:36871/kms/v1/keys
Aug 09 16:36:11 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 16:36:12 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
Aug 09 16:36:12 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293371, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
Aug 09 16:36:14 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:36:14.932320 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46795
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46795
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:36871/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:36:15.039650 17371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:36:15.039848 17371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:36:15.039870 17371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:36:15.040972 17371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:36:15.041006 17371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:36:15.041019 17371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:36:15.041030 17371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:36:15.041041 17371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:36:15.042650 17371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.15.212:36871/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46795
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:36:15.042848 17371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:36:15.043020 17371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:36:15.049153 17381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:15.049213 17383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:36:15.049237 17371 server_base.cc:1034] running on GCE node
W20260809 16:36:15.049166 17386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:36:15.049558 17371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:36:15.049868 17371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:36:15.051024 17371 hybrid_clock.cc:648] HybridClock initialized: now 1786293375051016 us; error 29 us; skew 500 ppm
Aug 09 16:36:15 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293375, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:36:15.057585 17371 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:36:15.058005 17371 webserver.cc:460] Webserver started at http://127.31.15.254:43297/ using document root <none> and password file <none>
I20260809 16:36:15.058218 17371 fs_manager.cc:359] Metadata directory not provided
I20260809 16:36:15.058256 17371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:36:15.058379 17371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:36:15 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293375, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:15.115103 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "7d87ae5fcd134e04b7eef34e844d178c"
format_stamp: "Formatted at 2026-08-09 16:36:15 on dist-test-slave-21b9"
server_key: "ab585f2bd4628da4fc0b336d86a54ee4"
server_key_iv: "0be617ae875a59c56760feccaa1360eb"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:15.115566 17371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "7d87ae5fcd134e04b7eef34e844d178c"
format_stamp: "Formatted at 2026-08-09 16:36:15 on dist-test-slave-21b9"
server_key: "ab585f2bd4628da4fc0b336d86a54ee4"
server_key_iv: "0be617ae875a59c56760feccaa1360eb"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:15.117180 17371 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:36:15.131286 17393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:15.131513 17371 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:36:15.131632 17371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "7d87ae5fcd134e04b7eef34e844d178c"
format_stamp: "Formatted at 2026-08-09 16:36:15 on dist-test-slave-21b9"
server_key: "ab585f2bd4628da4fc0b336d86a54ee4"
server_key_iv: "0be617ae875a59c56760feccaa1360eb"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:15.131723 17371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:36:15.148072 17371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:36:15.150097 17371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:36:15.150290 17371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:36:15.160192 17371 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:46795
I20260809 16:36:15.160279 17486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:46795 every 8 connection(s)
I20260809 16:36:15.160617 17371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:36:15.161801 17488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:36:15.166082 17488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: Bootstrap starting.
I20260809 16:36:15.167439 17488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: Neither blocks nor log segments found. Creating new log.
I20260809 16:36:15.168128 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 17371
I20260809 16:36:15.168177 17488 log.cc:824] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:36:15.168226 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:36:15.168406 31807 external_mini_cluster.cc:1428] Setting key 81727501fe48a78ed6211947ac8f64ce
I20260809 16:36:15.169094 17488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: No bootstrap required, opened a new log
I20260809 16:36:15.172956 17488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } }
I20260809 16:36:15.173098 17488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:36:15.173134 17488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d87ae5fcd134e04b7eef34e844d178c, State: Initialized, Role: FOLLOWER
I20260809 16:36:15.173287 17488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } }
I20260809 16:36:15.173355 17488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:36:15.173386 17488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:36:15.173445 17488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:36:15.173991 17488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } }
I20260809 16:36:15.174089 17488 leader_election.cc:302] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d87ae5fcd134e04b7eef34e844d178c; no voters:
I20260809 16:36:15.174288 17488 leader_election.cc:288] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:36:15.174403 17494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:36:15.174639 17488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:36:15.174661 17494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [term 1 LEADER]: Becoming Leader. State: Replica: 7d87ae5fcd134e04b7eef34e844d178c, State: Running, Role: LEADER
I20260809 16:36:15.174819 17494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } }
I20260809 16:36:15.175154 17494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d87ae5fcd134e04b7eef34e844d178c. Latest consensus state: current_term: 1 leader_uuid: "7d87ae5fcd134e04b7eef34e844d178c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } } }
I20260809 16:36:15.175225 17494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [sys.catalog]: This master's current role is: LEADER
I20260809 16:36:15.175559 17499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d87ae5fcd134e04b7eef34e844d178c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d87ae5fcd134e04b7eef34e844d178c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46795 } } }
I20260809 16:36:15.175695 17499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c [sys.catalog]: This master's current role is: LEADER
I20260809 16:36:15.176272 17488 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 16:36:15 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:36:15.179396 17507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:36:15.181078 17507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:36:15.184032 17507 catalog_manager.cc:1347] Generated new cluster ID: 91bf98681a524c558f2875f370482e26
I20260809 16:36:15.184082 17507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:36:15.185185 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:15.168964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.31.15.254:46795)
0809 16:36:15.169248 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:15.169259 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:15.169308 (+ 49us) server_negotiation.cc:407] Connection header received
0809 16:36:15.170335 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:15.170345 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:15.170424 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:15.170584 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:36:15.171402 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:15.173065 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:15.174220 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:15.174960 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:15.176699 (+ 1739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:15.179045 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:36:15.179157 (+ 112us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:36:15.179157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:36:15.179285 (+ 128us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:36:15.182870 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:15.183359 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:15.183361 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:15.183362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:15.183484 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:15.183736 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:15.183737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:15.183737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:15.184078 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:36:15.184712 (+ 634us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:15.184802 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:36:15.184957 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":113,"threads_started":1}
I20260809 16:36:15.201176 17507 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:36:15.201969 17507 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:36:15.204938 17507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: Generated new TSK 0
I20260809 16:36:15.205163 17507 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:36:16.066988 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:36871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:36:16.196460 17550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:36:16.196679 17550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:36:16.196712 17550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:36:16.197980 17550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:36:16.198024 17550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:36:16.198058 17550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:36:16.199661 17550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.15.212:36871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:36:16.199885 17550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:36:16.200121 17550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:36:16.206912 17561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.206910 17558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.207314 17563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:36:16.207513 17550 server_base.cc:1034] running on GCE node
I20260809 16:36:16.207731 17550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:36:16.208004 17550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:36:16.209216 17550 hybrid_clock.cc:648] HybridClock initialized: now 1786293376209197 us; error 32 us; skew 500 ppm
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:36:16.212783 17550 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:36:16.213413 17550 webserver.cc:460] Webserver started at http://127.31.15.193:44285/ using document root <none> and password file <none>
I20260809 16:36:16.213678 17550 fs_manager.cc:359] Metadata directory not provided
I20260809 16:36:16.213752 17550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:36:16.213892 17550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:16.231443 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "5c77d2995b02435ca311bb45c0684e18"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "ebc4aa63374db4a68a5f3b942c06f828"
server_key_iv: "938af7612548891410514c3eaafa5c62"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.233840 17550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "5c77d2995b02435ca311bb45c0684e18"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "ebc4aa63374db4a68a5f3b942c06f828"
server_key_iv: "938af7612548891410514c3eaafa5c62"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.239742 17550 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260809 16:36:16.255414 17571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.255856 17550 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 16:36:16.255972 17550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "5c77d2995b02435ca311bb45c0684e18"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "ebc4aa63374db4a68a5f3b942c06f828"
server_key_iv: "938af7612548891410514c3eaafa5c62"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.256083 17550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:36:16.279618 17550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:36:16.281585 17550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:36:16.281734 17550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:36:16.282014 17550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:36:16.282325 17550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:36:16.282359 17550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.282388 17550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:36:16.282413 17550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.296264 17550 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:44783
I20260809 16:36:16.296535 17769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:44783 every 8 connection(s)
I20260809 16:36:16.296743 17550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:36:16.304548 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 17550
I20260809 16:36:16.304666 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:36:16.304881 31807 external_mini_cluster.cc:1428] Setting key c1ee80491d679e8ca07511be062cd202
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:36:16.308985 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.297513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46795 (local address 127.31.15.193:39335)
0809 16:36:16.297835 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.297887 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.298919 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.299187 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.299193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.299600 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:36:16.300022 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.300028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.300720 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.300721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.302152 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.302155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.302668 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.302670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.302874 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.303546 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:36:16.303556 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:36:16.305400 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:36:16.307314 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.307322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.307324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.307637 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.307872 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.307872 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.307873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.308014 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.308310 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:36:16.308317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:36:16.308493 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.308589 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.308737 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":173,"threads_started":1}
I20260809 16:36:16.309652 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.297577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39335 (local address 127.31.15.254:46795)
0809 16:36:16.297737 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.297739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.298026 (+ 287us) server_negotiation.cc:407] Connection header received
0809 16:36:16.299042 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.299043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.299055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.299107 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:36:16.300104 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.300629 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.302253 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.302569 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.302647 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.305495 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:36:16.305497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:36:16.305497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:36:16.305503 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:36:16.307189 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.307725 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.307727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.307727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.307799 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.308091 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.308092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.308092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.308227 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:36:16.308283 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.309482 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.309568 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260809 16:36:16.310374 17770 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46795
I20260809 16:36:16.310477 17770 heartbeater.cc:464] Registering TS with master...
I20260809 16:36:16.310653 17770 heartbeater.cc:511] Master 127.31.15.254:46795 requested a full tablet report, sending...
I20260809 16:36:16.311256 17426 ts_manager.cc:194] Registered new tserver with Master: 5c77d2995b02435ca311bb45c0684e18 (127.31.15.193:44783)
I20260809 16:36:16.312173 17426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:39335
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:36:16.331945 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:36871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:36:16.489764 17782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:36:16.490043 17782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:36:16.490087 17782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:36:16.491930 17782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:36:16.491990 17782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:36:16.492030 17782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:36:16.494251 17782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.15.212:36871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:36:16.494522 17782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:36:16.494846 17782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 16:36:16.503244 17802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.503309 17794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.503592 17797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:36:16.503706 17782 server_base.cc:1034] running on GCE node
I20260809 16:36:16.503952 17782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:36:16.504192 17782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:36:16.505347 17782 hybrid_clock.cc:648] HybridClock initialized: now 1786293376505331 us; error 34 us; skew 500 ppm
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:36:16.508388 17782 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:36:16.508826 17782 webserver.cc:460] Webserver started at http://127.31.15.194:44613/ using document root <none> and password file <none>
I20260809 16:36:16.509076 17782 fs_manager.cc:359] Metadata directory not provided
I20260809 16:36:16.509136 17782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:36:16.509274 17782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:16.520864 17782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "610432d2350c46939f5ed9adb23e07d7"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "be972d40ad420b2e2ce39437653e86f1"
server_key_iv: "0a5ede82a4c9c80f767decae9360d727"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.521407 17782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "610432d2350c46939f5ed9adb23e07d7"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "be972d40ad420b2e2ce39437653e86f1"
server_key_iv: "0a5ede82a4c9c80f767decae9360d727"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.524502 17782 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 16:36:16.544211 17812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.544401 17782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 16:36:16.544507 17782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "610432d2350c46939f5ed9adb23e07d7"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "be972d40ad420b2e2ce39437653e86f1"
server_key_iv: "0a5ede82a4c9c80f767decae9360d727"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.544600 17782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:36:16.570670 17782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:36:16.572774 17782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:36:16.572933 17782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:36:16.573158 17782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:36:16.573556 17782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:36:16.573601 17782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.573630 17782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:36:16.573653 17782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.589553 17782 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:44133
I20260809 16:36:16.589761 17988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:44133 every 8 connection(s)
I20260809 16:36:16.590162 17782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:36:16.599661 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 17782
I20260809 16:36:16.599772 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:36:16.600148 31807 external_mini_cluster.cc:1428] Setting key 94bd076a8768210406c9be1d4f14acdb
I20260809 16:36:16.602844 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.591210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:50855 (local address 127.31.15.254:46795)
0809 16:36:16.591258 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.591260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.591749 (+ 489us) server_negotiation.cc:407] Connection header received
0809 16:36:16.592724 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.592726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.592745 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.592776 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:36:16.593985 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.594784 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.596322 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.596704 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.596764 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.599597 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:36:16.599598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:36:16.599598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:36:16.599606 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:36:16.601102 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.601604 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.601605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.601605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.601670 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.602106 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.602107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.602107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.602234 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:36:16.602292 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.602659 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.602755 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:36:16.603626 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.591206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46795 (local address 127.31.15.194:50855)
0809 16:36:16.591573 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.591623 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.592592 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.593033 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.593039 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.593441 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:36:16.593851 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.593859 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.594905 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.594906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.596236 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.596239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.596794 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.596795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.596966 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.597544 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:36:16.597554 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:36:16.599469 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:36:16.601189 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.601196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.601199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.601515 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.601729 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.601730 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.601730 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.601875 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.602303 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:36:16.602310 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:36:16.602473 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.603166 (+ 693us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.603362 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":202,"threads_started":1}
I20260809 16:36:16.604176 17989 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46795
I20260809 16:36:16.604287 17989 heartbeater.cc:464] Registering TS with master...
I20260809 16:36:16.604492 17989 heartbeater.cc:511] Master 127.31.15.254:46795 requested a full tablet report, sending...
I20260809 16:36:16.604894 17426 ts_manager.cc:194] Registered new tserver with Master: 610432d2350c46939f5ed9adb23e07d7 (127.31.15.194:44133)
I20260809 16:36:16.605365 17426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:50855
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:36:16.632210 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:36871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:36:16.808861 17997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:36:16.809190 17997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:36:16.809238 17997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:36:16.811182 17997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:36:16.811246 17997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:36:16.811322 17997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:36:16.813836 17997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.15.212:36871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:36:16.814100 17997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:36:16.814400 17997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:36:16.822248 18006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.822655 18004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:36:16.823153 18010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:36:16.828577 17997 server_base.cc:1034] running on GCE node
I20260809 16:36:16.828898 17997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:36:16.829248 17997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:36:16.830399 17997 hybrid_clock.cc:648] HybridClock initialized: now 1786293376830385 us; error 29 us; skew 500 ppm
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:36:16.833139 17997 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:36:16.833550 17997 webserver.cc:460] Webserver started at http://127.31.15.195:35377/ using document root <none> and password file <none>
I20260809 16:36:16.833726 17997 fs_manager.cc:359] Metadata directory not provided
I20260809 16:36:16.833765 17997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:36:16.833863 17997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:16.857295 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "d8f0c4a69aaf45ed920967435378352a"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "53613feb553f985402dd5c7ad633a083"
server_key_iv: "c585cdab2f767ba74b21eecb91a78652"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.857836 17997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "d8f0c4a69aaf45ed920967435378352a"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "53613feb553f985402dd5c7ad633a083"
server_key_iv: "c585cdab2f767ba74b21eecb91a78652"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.859934 17997 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:36:16.879294 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.879853 17997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:36:16.879973 17997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "d8f0c4a69aaf45ed920967435378352a"
format_stamp: "Formatted at 2026-08-09 16:36:16 on dist-test-slave-21b9"
server_key: "53613feb553f985402dd5c7ad633a083"
server_key_iv: "c585cdab2f767ba74b21eecb91a78652"
server_key_version: "kuduclusterkey@0"
I20260809 16:36:16.880069 17997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:36:16.903712 17997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:36:16.905895 17997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:36:16.906062 17997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:36:16.906477 17997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:36:16.906925 17997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:36:16.906976 17997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.907008 17997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:36:16.907024 17997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:36:16.928375 17997 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:37477
I20260809 16:36:16.928517 18208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:37477 every 8 connection(s)
I20260809 16:36:16.928874 17997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:36:16 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:36:16.938261 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 17997
I20260809 16:36:16.938364 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:36:16.938531 31807 external_mini_cluster.cc:1428] Setting key 794b15c17f15b27e28f77650fc198aa9
I20260809 16:36:16.940131 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.930020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:45199 (local address 127.31.15.254:46795)
0809 16:36:16.930093 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.930096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.930477 (+ 381us) server_negotiation.cc:407] Connection header received
0809 16:36:16.931292 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.931294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.931307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.931395 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:36:16.932283 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.933062 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.934252 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.934556 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.934636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.937164 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:36:16.937165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:36:16.937166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:36:16.937175 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:36:16.938702 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.939185 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.939186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.939186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.939254 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.939523 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.939524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.939525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.939640 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:36:16.939696 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.939952 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.940043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:36:16.940349 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.929954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46795 (local address 127.31.15.195:45199)
0809 16:36:16.930317 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.930362 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.931165 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.931453 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.931456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.931860 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:36:16.932190 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.932195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.933167 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.933168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.934160 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.934163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.934634 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.934635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.934774 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.935415 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:36:16.935424 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:36:16.937078 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:36:16.938800 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.938808 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.938811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.939113 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.939331 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:36:16.939332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:36:16.939333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:36:16.939463 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:36:16.939695 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:36:16.939702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:36:16.939834 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.939923 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.940066 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":235,"threads_started":1}
I20260809 16:36:16.940925 18209 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46795
I20260809 16:36:16.941022 18209 heartbeater.cc:464] Registering TS with master...
I20260809 16:36:16.941197 18209 heartbeater.cc:511] Master 127.31.15.254:46795 requested a full tablet report, sending...
I20260809 16:36:16.941582 17426 ts_manager.cc:194] Registered new tserver with Master: d8f0c4a69aaf45ed920967435378352a (127.31.15.195:37477)
I20260809 16:36:16.941926 17426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:45199
I20260809 16:36:16.950121 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:36:16.962715 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.951622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.31.15.254:46795)
0809 16:36:16.951664 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.951666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.952031 (+ 365us) server_negotiation.cc:407] Connection header received
0809 16:36:16.952126 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.952127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.952137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.952164 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:36:16.956395 (+ 4231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.957099 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.958294 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.958566 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.958603 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.960178 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:36:16.960179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:36:16.960179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:36:16.960187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:36:16.961644 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.962023 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.962024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.962024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.962080 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:36:16.962256 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:36:16.962257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:36:16.962257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:36:16.962364 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:36:16.962450 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.962551 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.962623 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:36:16.964351 17426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:36:16.966722 17426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:36:16.978950 18226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.970945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:44133 (local address 127.0.0.1:35036)
0809 16:36:16.974344 (+ 3399us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.974355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.974423 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.974710 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.974713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.974715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:36:16.974781 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.974783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.976228 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.976229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.977574 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.977576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.978641 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.978642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.978724 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.978738 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.978809 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.978862 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3353,"thread_start_us":156,"threads_started":1}
I20260809 16:36:16.979420 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.971347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.31.15.194:44133)
0809 16:36:16.972249 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.972255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.974418 (+ 2163us) server_negotiation.cc:407] Connection header received
0809 16:36:16.974519 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.974522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.974540 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.974616 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:36:16.974859 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.976139 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.977687 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.978441 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.978523 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.978603 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.979257 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.979339 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":161,"threads_started":1}
I20260809 16:36:16.980396 17862 tablet_service.cc:1467] Processing CreateTablet for tablet 8551f7822d2b4ce29a96590366591cd0 (DEFAULT_TABLE table=test-table [id=29b6ee78f2cb4a99aebb86395cc4acdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:36:16.980787 17862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8551f7822d2b4ce29a96590366591cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:36:16.981647 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.971056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.31.15.193:44783)
0809 16:36:16.974159 (+ 3103us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.974164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.974176 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:36:16.974544 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.974547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.974564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.974643 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:36:16.974988 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.976316 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.977918 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.978745 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.978814 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.978870 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.981467 (+ 2597us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.981528 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3064,"thread_start_us":50,"threads_started":1}
I20260809 16:36:16.984760 18231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.971219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:37477 (local address 127.0.0.1:37372)
0809 16:36:16.974879 (+ 3660us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.974887 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.974924 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.979929 (+ 5005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.979930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.979932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:36:16.979969 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.979970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.981834 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.981835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.983237 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.983240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.984555 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.984556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.984616 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.984621 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.984651 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.984687 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3644,"thread_start_us":138,"threads_started":1}
I20260809 16:36:16.985563 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.971524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37372 (local address 127.31.15.195:37477)
0809 16:36:16.979620 (+ 8096us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:16.979625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:16.979636 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:36:16.979708 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:16.979710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:16.979727 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:16.979806 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:36:16.980052 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.981368 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.983484 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.984445 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.985064 (+ 619us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.985297 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:16.985327 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 16:36:16.985415 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8007,"thread_start_us":72,"threads_started":1}
I20260809 16:36:16.979131 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:16.970943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:44783 (local address 127.0.0.1:45518)
0809 16:36:16.971685 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:16.971728 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:16.971799 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:16.974723 (+ 2924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:16.974724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:16.974727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:36:16.974782 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.974784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.976735 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.976736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.977821 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:16.977824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:16.978904 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:16.978905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:16.978965 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:16.978969 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:36:16.979020 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:36:16.979058 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"thread_start_us":108,"threads_started":1}
I20260809 16:36:16.986443 18105 tablet_service.cc:1467] Processing CreateTablet for tablet 8551f7822d2b4ce29a96590366591cd0 (DEFAULT_TABLE table=test-table [id=29b6ee78f2cb4a99aebb86395cc4acdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:36:16.986560 17660 tablet_service.cc:1467] Processing CreateTablet for tablet 8551f7822d2b4ce29a96590366591cd0 (DEFAULT_TABLE table=test-table [id=29b6ee78f2cb4a99aebb86395cc4acdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:36:16.986884 17660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8551f7822d2b4ce29a96590366591cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:36:16.986887 18105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8551f7822d2b4ce29a96590366591cd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:36:16.990880 18236 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Bootstrap starting.
I20260809 16:36:16.992259 18236 tablet_bootstrap.cc:654] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Neither blocks nor log segments found. Creating new log.
I20260809 16:36:16.993072 18236 log.cc:824] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Log is configured to *not* fsync() on all Append() calls
I20260809 16:36:16.993505 18235 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Bootstrap starting.
I20260809 16:36:16.994894 18235 tablet_bootstrap.cc:654] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Neither blocks nor log segments found. Creating new log.
I20260809 16:36:16.995018 18236 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: No bootstrap required, opened a new log
I20260809 16:36:16.995129 18236 ts_tablet_manager.cc:1397] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 16:36:16.995502 18235 log.cc:824] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:36:16.996245 18235 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: No bootstrap required, opened a new log
I20260809 16:36:16.996315 18235 ts_tablet_manager.cc:1397] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 16:36:16.999455 18237 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Bootstrap starting.
I20260809 16:36:16.999641 18235 raft_consensus.cc:348] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:16.999804 18235 raft_consensus.cc:374] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:36:16.999841 18235 raft_consensus.cc:729] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 610432d2350c46939f5ed9adb23e07d7, State: Initialized, Role: FOLLOWER
I20260809 16:36:16.999994 18235 consensus_queue.cc:260] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.000286 18235 ts_tablet_manager.cc:1428] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 16:36:17.000408 17989 heartbeater.cc:503] Master 127.31.15.254:46795 was elected leader, sending a full tablet report...
I20260809 16:36:17.000855 18237 tablet_bootstrap.cc:654] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Neither blocks nor log segments found. Creating new log.
I20260809 16:36:17.001330 18237 log.cc:824] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:36:17.007797 18237 tablet_bootstrap.cc:492] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: No bootstrap required, opened a new log
I20260809 16:36:17.007879 18237 ts_tablet_manager.cc:1397] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260809 16:36:17.011536 18237 raft_consensus.cc:348] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.011740 18237 raft_consensus.cc:374] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:36:17.011786 18237 raft_consensus.cc:729] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f0c4a69aaf45ed920967435378352a, State: Initialized, Role: FOLLOWER
I20260809 16:36:17.011966 18237 consensus_queue.cc:260] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.012306 18237 ts_tablet_manager.cc:1428] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 16:36:17.012441 18209 heartbeater.cc:503] Master 127.31.15.254:46795 was elected leader, sending a full tablet report...
I20260809 16:36:17.012537 18236 raft_consensus.cc:348] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.012760 18236 raft_consensus.cc:374] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:36:17.012842 18236 raft_consensus.cc:729] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c77d2995b02435ca311bb45c0684e18, State: Initialized, Role: FOLLOWER
I20260809 16:36:17.013007 18236 consensus_queue.cc:260] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.013423 18236 ts_tablet_manager.cc:1428] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Time spent starting tablet: real 0.018s user 0.000s sys 0.006s
I20260809 16:36:17.013641 17770 heartbeater.cc:503] Master 127.31.15.254:46795 was elected leader, sending a full tablet report...
W20260809 16:36:17.047845 17771 tablet.cc:2369] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:36:17.067399 18244 raft_consensus.cc:482] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:36:17.067575 18244 raft_consensus.cc:504] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.068063 18244 leader_election.cc:288] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 610432d2350c46939f5ed9adb23e07d7 (127.31.15.194:44133), 5c77d2995b02435ca311bb45c0684e18 (127.31.15.193:44783)
I20260809 16:36:17.072288 18212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:17.068349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:44783 (local address 127.31.15.195:57437)
0809 16:36:17.068394 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:17.068407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:17.068458 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:17.068637 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:17.068638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:17.068639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:36:17.068763 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:17.068766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.069693 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.069694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:17.070860 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:17.070862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.071984 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.071985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:17.072123 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:17.072127 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:36:17.072174 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:36:17.072213 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 16:36:17.072793 18230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:17.068490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:57437 (local address 127.31.15.193:44783)
0809 16:36:17.068531 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:17.068533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:17.068540 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:36:17.068568 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:17.068569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:17.068577 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:17.068623 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:36:17.068879 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.069594 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.070979 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.071876 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.071944 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:17.071954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:17.072680 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 16:36:17.072726 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:36:17.073483 17670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8551f7822d2b4ce29a96590366591cd0" candidate_uuid: "d8f0c4a69aaf45ed920967435378352a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c77d2995b02435ca311bb45c0684e18" is_pre_election: true
I20260809 16:36:17.073644 17670 raft_consensus.cc:2454] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f0c4a69aaf45ed920967435378352a in term 0.
I20260809 16:36:17.073804 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:17.069415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:55405 (local address 127.31.15.194:44133)
0809 16:36:17.069459 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:17.069461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:17.069788 (+ 327us) server_negotiation.cc:407] Connection header received
0809 16:36:17.069881 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:17.069882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:17.069896 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:17.069924 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:36:17.070371 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.071085 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.072616 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.073187 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.073234 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:17.073242 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:17.073699 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 16:36:17.073739 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:36:17.073889 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:17.068884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:44133 (local address 127.31.15.195:55405)
0809 16:36:17.069578 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:17.069588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:17.069814 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:17.069994 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:36:17.069995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:36:17.069997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:36:17.070278 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:17.070280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.071229 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.071230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:17.072534 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:36:17.072537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.073451 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.073452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:36:17.073547 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:17.073551 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:36:17.073606 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:36:17.073643 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":426,"threads_started":1}
I20260809 16:36:17.074154 18024 leader_election.cc:302] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c77d2995b02435ca311bb45c0684e18, d8f0c4a69aaf45ed920967435378352a; no voters:
I20260809 16:36:17.074306 18244 raft_consensus.cc:2781] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:36:17.074379 18244 raft_consensus.cc:482] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:36:17.074405 18244 raft_consensus.cc:3037] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:36:17.074524 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8551f7822d2b4ce29a96590366591cd0" candidate_uuid: "d8f0c4a69aaf45ed920967435378352a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610432d2350c46939f5ed9adb23e07d7" is_pre_election: true
I20260809 16:36:17.074729 17917 raft_consensus.cc:2454] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f0c4a69aaf45ed920967435378352a in term 0.
I20260809 16:36:17.075932 18244 raft_consensus.cc:504] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.076062 18244 leader_election.cc:288] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [CANDIDATE]: Term 1 election: Requested vote from peers 610432d2350c46939f5ed9adb23e07d7 (127.31.15.194:44133), 5c77d2995b02435ca311bb45c0684e18 (127.31.15.193:44783)
I20260809 16:36:17.076229 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8551f7822d2b4ce29a96590366591cd0" candidate_uuid: "d8f0c4a69aaf45ed920967435378352a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "610432d2350c46939f5ed9adb23e07d7"
I20260809 16:36:17.076263 17670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8551f7822d2b4ce29a96590366591cd0" candidate_uuid: "d8f0c4a69aaf45ed920967435378352a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c77d2995b02435ca311bb45c0684e18"
I20260809 16:36:17.076301 17917 raft_consensus.cc:3037] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:36:17.076340 17670 raft_consensus.cc:3037] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:36:17.077231 17670 raft_consensus.cc:2454] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f0c4a69aaf45ed920967435378352a in term 1.
I20260809 16:36:17.077425 18024 leader_election.cc:302] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c77d2995b02435ca311bb45c0684e18, d8f0c4a69aaf45ed920967435378352a; no voters:
I20260809 16:36:17.077502 18244 raft_consensus.cc:2781] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:36:17.077837 17917 raft_consensus.cc:2454] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f0c4a69aaf45ed920967435378352a in term 1.
I20260809 16:36:17.077883 18244 raft_consensus.cc:686] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 1 LEADER]: Becoming Leader. State: Replica: d8f0c4a69aaf45ed920967435378352a, State: Running, Role: LEADER
I20260809 16:36:17.077991 18244 consensus_queue.cc:237] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } }
I20260809 16:36:17.078877 17426 catalog_manager.cc:5495] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8f0c4a69aaf45ed920967435378352a (127.31.15.195). New cstate: current_term: 1 leader_uuid: "d8f0c4a69aaf45ed920967435378352a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f0c4a69aaf45ed920967435378352a" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 37477 } health_report { overall_health: HEALTHY } } }
W20260809 16:36:17.094717 17990 tablet.cc:2369] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:36:17.103319 18234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:36:17.099911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37386 (local address 127.31.15.195:37477)
0809 16:36:17.099952 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:36:17.099954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:36:17.099984 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:36:17.100024 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:36:17.100025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:36:17.100046 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:36:17.100109 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:36:17.100353 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.101030 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.102069 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:36:17.102354 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:36:17.102414 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:36:17.102605 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:36:17.102732 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:36:17.102796 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:36:17.103192 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 16:36:17.103214 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:36:17.105804 17670 raft_consensus.cc:1261] T 8551f7822d2b4ce29a96590366591cd0 P 5c77d2995b02435ca311bb45c0684e18 [term 1 FOLLOWER]: Refusing update from remote peer d8f0c4a69aaf45ed920967435378352a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:36:17.106132 17917 raft_consensus.cc:1261] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 1 FOLLOWER]: Refusing update from remote peer d8f0c4a69aaf45ed920967435378352a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:36:17.106228 18248 consensus_queue.cc:1035] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c77d2995b02435ca311bb45c0684e18" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 44783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:36:17.106529 18248 consensus_queue.cc:1035] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [LEADER]: Connected to new peer: Peer: permanent_uuid: "610432d2350c46939f5ed9adb23e07d7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:36:17.113037 18251 mvcc.cc:205] Tried to move back new op lower bound from 7316657672621903872 to 7316657672512151552. Current Snapshot: MvccSnapshot[applied={T|T < 7316657672621903872}]
I20260809 16:36:17.113270 18253 mvcc.cc:205] Tried to move back new op lower bound from 7316657672621903872 to 7316657672512151552. Current Snapshot: MvccSnapshot[applied={T|T < 7316657672621903872}]
I20260809 16:36:17.113448 18257 mvcc.cc:205] Tried to move back new op lower bound from 7316657672621903872 to 7316657672512151552. Current Snapshot: MvccSnapshot[applied={T|T < 7316657672621903872}]
I20260809 16:36:17.117305 17425 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:36:17.117398 17425 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:36:17.119704 17425 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d87ae5fcd134e04b7eef34e844d178c: Sending DeleteTablet for 3 replicas of tablet 8551f7822d2b4ce29a96590366591cd0
I20260809 16:36:17.120301 17862 tablet_service.cc:1514] Processing DeleteTablet for tablet 8551f7822d2b4ce29a96590366591cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:36:17 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:35036
I20260809 16:36:17.120580 17660 tablet_service.cc:1514] Processing DeleteTablet for tablet 8551f7822d2b4ce29a96590366591cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:36:17 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45518
I20260809 16:36:17.121192 18105 tablet_service.cc:1514] Processing DeleteTablet for tablet 8551f7822d2b4ce29a96590366591cd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:36:17 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:37372
I20260809 16:36:17.121608 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 17550
I20260809 16:36:17.133379 18270 tablet_replica.cc:330] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: stopping tablet replica
I20260809 16:36:17.133574 18268 tablet_replica.cc:330] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: stopping tablet replica
I20260809 16:36:17.133761 18268 raft_consensus.cc:2229] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:36:17.133920 18270 raft_consensus.cc:2229] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:36:17.134122 18268 raft_consensus.cc:2258] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:36:17.134307 18270 raft_consensus.cc:2258] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:36:17.134958 18270 ts_tablet_manager.cc:1905] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:36:17.138072 18270 ts_tablet_manager.cc:1918] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:36:17.138199 18270 log.cc:1196] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/8551f7822d2b4ce29a96590366591cd0
I20260809 16:36:17.138581 18270 ts_tablet_manager.cc:1939] T 8551f7822d2b4ce29a96590366591cd0 P d8f0c4a69aaf45ed920967435378352a: Deleting consensus metadata
I20260809 16:36:17.139061 17398 catalog_manager.cc:4841] TS d8f0c4a69aaf45ed920967435378352a (127.31.15.195:37477): tablet 8551f7822d2b4ce29a96590366591cd0 (table test-table [id=29b6ee78f2cb4a99aebb86395cc4acdc]) successfully deleted
W20260809 16:36:17.139674 17395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:44783 (error 108)
I20260809 16:36:17.140571 18268 ts_tablet_manager.cc:1905] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 16:36:17 dist-test-slave-21b9 krb5kdc[16669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293376, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
W20260809 16:36:17.141088 18226 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:36:17.140081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:44783 (local address 127.0.0.1:45524)
0809 16:36:17.140133 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:36:17.140146 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:36:17.140199 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:36:17.141039 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:44783: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:36:17.141201 17395 catalog_manager.cc:4584] TS 5c77d2995b02435ca311bb45c0684e18 (127.31.15.193:44783): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8551f7822d2b4ce29a96590366591cd0: Network error: Client connection negotiation failed: client connection to 127.31.15.193:44783: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:36:17.142457 18268 ts_tablet_manager.cc:1918] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:36:17.142534 18268 log.cc:1196] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/8551f7822d2b4ce29a96590366591cd0
I20260809 16:36:17.142771 18268 ts_tablet_manager.cc:1939] T 8551f7822d2b4ce29a96590366591cd0 P 610432d2350c46939f5ed9adb23e07d7: Deleting consensus metadata
I20260809 16:36:17.143261 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 17782
I20260809 16:36:17.143714 17400 catalog_manager.cc:4841] TS 610432d2350c46939f5ed9adb23e07d7 (127.31.15.194:44133): tablet 8551f7822d2b4ce29a96590366591cd0 (table test-table [id=29b6ee78f2cb4a99aebb86395cc4acdc]) successfully deleted
I20260809 16:36:17.155108 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 17997
I20260809 16:36:17.172211 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 17371
I20260809 16:36:17.199496 31807 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:36:17.546303 31807 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:36:17.546425 31807 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:36:17.889911 31807 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:36:17.890 UTC [16723] LOG: received smart shutdown request
2026-08-09 16:36:17.893 UTC [16723] LOG: background worker "logical replication launcher" (PID 16736) exited with exit code 1
2026-08-09 16:36:17.893 UTC [16731] LOG: shutting down
2026-08-09 16:36:18.030 UTC [16723] LOG: database system is shut down
2026-08-09T16:36:18Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46190 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:36:18 dist-test-slave-21b9 krb5kdc[18293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:36:18 dist-test-slave-21b9 krb5kdc[18293](info): set up 2 sockets
Aug 09 16:36:18 dist-test-slave-21b9 krb5kdc[18293](info): commencing operation
krb5kdc: starting...
W20260809 16:36:19.284549 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:36:19 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:36:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangeradmin_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangeradmin_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangerlookup_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangerlookup_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
I20260809 16:36:19.434906 31807 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskfvbJG8/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/postgres -l logfile start
2026-08-09 16:36:21.670 UTC [18343] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 16:36:21.670 UTC [18343] LOG: listening on IPv4 address "127.31.15.212", port 55817
2026-08-09 16:36:21.673 UTC [18343] LOG: listening on Unix socket "/tmp/.s.PGSQL.55817"
2026-08-09 16:36:21.693 UTC [18348] LOG: database system was shut down at 2026-08-09 16:36:20 UTC
2026-08-09 16:36:21.697 UTC [18343] LOG: database system is ready to accept connections
I20260809 16:36:22.752879 31807 mini_postgres.cc:96] Postgres bound to 55817
2026-08-09 16:36:22.759 UTC [18358] FATAL: database "slave" does not exist
127.31.15.212:55817 - accepting connections
I20260809 16:36:22.759979 31807 mini_ranger.cc:161] Starting Ranger...
I20260809 16:36:22.774168 31807 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 16:36:23.182587 31807 mini_ranger.cc:87] Created ranger Postgres database
I20260809 16:36:23.182677 31807 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin
2026-08-09 16:36:23,455 [I] DB FLAVOR :POSTGRES
2026-08-09 16:36:23,456 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:36:23,456 [I] Checking connection..
2026-08-09 16:36:23,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 16:36:23,686 [I] Checking connection passed.
2026-08-09 16:36:23,686 [I] --------- Verifying version history table ---------
2026-08-09 16:36:23,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:36:23,942 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:36:23,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:36:24,185 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 16:36:24,186 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 16:36:24,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 16:36:24.429 UTC [18464] WARNING: there is no transaction in progress
2026-08-09 16:36:24,437 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 16:36:24,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 16:36:24,674 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 16:36:24,674 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:36:24,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 16:36:24,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 16:36:25,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-21b9.c.gcp-upstream.internal','N') ;"
2026-08-09 16:36:25,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:36:25,650 [I] Table x_portal_user does not exist in database ranger
2026-08-09 16:36:25,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:36:25,897 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 16:36:25,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:36:26,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:36:26,414 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 16:36:26,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 16:36:27.305 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.319 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.332 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.345 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.359 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.375 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.424 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.430 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.443 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.452 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.461 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.474 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.482 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.488 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.493 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:27.498 UTC [18652] WARNING: there is no transaction in progress
2026-08-09 16:36:28,049 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 16:36:28,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 16:36:28,290 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 16:36:28,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 16:36:28,549 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 16:36:28,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 16:36:28,809 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 16:36:28,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 16:36:29,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 16:36:29,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-21b9.c.gcp-upstream.internal';"
2026-08-09 16:36:29,562 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 16:36:29,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 16:36:29,816 [I] DB_PATCHES have already been applied
I20260809 16:36:29.821920 31807 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskfvbJG8/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 16:36:29.821998 31807 mini_ranger.cc:193] Using host: 127.31.15.212
I20260809 16:36:29.825724 31807 mini_ranger.cc:227] Ranger admin URL: http://127.31.15.212:33549
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:36:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 16:36:31.061126 31807 mini_ranger.cc:160] Time spent starting Ranger: real 8.301s user 0.001s sys 0.008s
Aug 09, 2026 4:36:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:36:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 4:36:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:36:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:36:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 16:36:52.124047 31807 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.31.15.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.31.15.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangerkms_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.31.15.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/rangerkms_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.31.15.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.31.15.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.31.15.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/HTTP_127.31.15.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 16:36:52 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293412, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 16:36:52.223786 31807 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 16:36:52.234282 31807 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 16:36:52.940608 31807 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 16:36:52.940678 31807 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms
I20260809 16:36:52.940690 31807 mini_ranger_kms.cc:227] Using postgres at 127.31.15.212:55817
2026-08-09 16:36:53,104 [I] DB FLAVOR :POSTGRES
2026-08-09 16:36:53,104 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 16:36:53,104 [I] Checking connection
2026-08-09 16:36:53,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 16:36:53,330 [I] connection success
2026-08-09 16:36:53,330 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 16:36:53,330 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 16:36:53,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 16:36:53,563 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 16:36:53,563 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 16:36:53,563 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 16:36:53,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.31.15.212:55817/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 16:36:53,822 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 16:36:54.268220 31807 mini_ranger_kms.cc:310] Created kms service
I20260809 16:36:54.478822 31807 mini_ranger_kms.cc:326] Created kudu user
I20260809 16:36:54.519362 31807 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 16:36:54 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293412, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:36:54.924415 31807 mini_ranger_kms.cc:384] Added ranger policy
I20260809 16:36:54.924661 31807 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms:/tmp/dist-test-taskfvbJG8/build/release/bin/postgresql.jar:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskfvbJG8/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 16:36:54.924693 31807 mini_ranger_kms.cc:240] Using host: 127.31.15.212
I20260809 16:36:54.928709 31807 mini_ranger_kms.cc:276] Ranger KMS PID: 18950
I20260809 16:36:54.928787 31807 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.31.15.212:34927
Aug 09, 2026 4:36:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:36:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:36:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:36:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:36:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:36:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 16:36:57 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293417, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:36:57.451546 31807 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.228s user 0.007s sys 0.141s
I20260809 16:36:57.451633 31807 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 16:36:57.451681 31807 mini_ranger_kms.cc:401] 127.31.15.212:34927/kms/v1/keys
Aug 09 16:36:57 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293417, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
Aug 09 16:36:57 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293417, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.31.15.212@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
Aug 09 16:37:00 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:37:00.366246 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:41555
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:41555
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:00.465430 19042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:00.465624 19042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:00.465644 19042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:00.466765 19042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:00.466809 19042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:00.466821 19042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:00.466831 19042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:00.466843 19042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:00.468446 19042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
--ranger_kms_url=127.31.15.212:34927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:41555
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:00.468675 19042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:00.468904 19042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:00.473904 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:00.474048 19042 server_base.cc:1034] running on GCE node
W20260809 16:37:00.473910 19049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:00.474186 19051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:00.474470 19042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:00.474817 19042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:00.475987 19042 hybrid_clock.cc:648] HybridClock initialized: now 1786293420475968 us; error 34 us; skew 500 ppm
Aug 09 16:37:00 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293420, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:00.478950 19042 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:00.479485 19042 webserver.cc:460] Webserver started at http://127.31.15.254:34975/ using document root <none> and password file <none>
I20260809 16:37:00.479734 19042 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:00.479784 19042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:00.479924 19042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:37:00 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293420, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:00.532770 19042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "ca9b3a8d082a45508c0433ed614efd6f"
format_stamp: "Formatted at 2026-08-09 16:37:00 on dist-test-slave-21b9"
server_key: "4e2e8b9708a27dc543f01c4f817dc7e5"
server_key_iv: "13a5234823fd7996a1e41cb2c734c6c7"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:00.533239 19042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "ca9b3a8d082a45508c0433ed614efd6f"
format_stamp: "Formatted at 2026-08-09 16:37:00 on dist-test-slave-21b9"
server_key: "4e2e8b9708a27dc543f01c4f817dc7e5"
server_key_iv: "13a5234823fd7996a1e41cb2c734c6c7"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:00.535017 19042 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 16:37:00.549021 19069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:00.549256 19042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:00.549356 19042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ca9b3a8d082a45508c0433ed614efd6f"
format_stamp: "Formatted at 2026-08-09 16:37:00 on dist-test-slave-21b9"
server_key: "4e2e8b9708a27dc543f01c4f817dc7e5"
server_key_iv: "13a5234823fd7996a1e41cb2c734c6c7"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:00.549453 19042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:00.569149 19042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:00.571157 19042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:00.571354 19042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:00.576474 19042 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:41555
I20260809 16:37:00.576508 19148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:41555 every 8 connection(s)
I20260809 16:37:00.576927 19042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:00.577684 19150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:00.580014 19150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Bootstrap starting.
I20260809 16:37:00.580600 19150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:00.580862 19150 log.cc:824] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:00.581501 19150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: No bootstrap required, opened a new log
I20260809 16:37:00.583293 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19042
I20260809 16:37:00.583403 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:37:00.583598 31807 external_mini_cluster.cc:1428] Setting key 6404a1bd228857ef69da3665ab57edcf
I20260809 16:37:00.583705 19150 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:00.583874 19150 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:00.583912 19150 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9b3a8d082a45508c0433ed614efd6f, State: Initialized, Role: FOLLOWER
I20260809 16:37:00.584106 19150 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:00.584280 19150 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:00.584370 19150 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:00.584430 19150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:00.585062 19150 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:00.585160 19150 leader_election.cc:302] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca9b3a8d082a45508c0433ed614efd6f; no voters:
I20260809 16:37:00.585352 19150 leader_election.cc:288] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:37:00.585736 19150 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:00.585388 19156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:00.587038 19156 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 LEADER]: Becoming Leader. State: Replica: ca9b3a8d082a45508c0433ed614efd6f, State: Running, Role: LEADER
I20260809 16:37:00.587139 19156 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:00.587750 19150 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:37:00.587729 19158 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca9b3a8d082a45508c0433ed614efd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } } }
I20260809 16:37:00.587802 19158 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:00.588129 19177 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9b3a8d082a45508c0433ed614efd6f. Latest consensus state: current_term: 1 leader_uuid: "ca9b3a8d082a45508c0433ed614efd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } } }
I20260809 16:37:00.590590 19177 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:00.592329 19180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:00.592903 19180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 16:37:00 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:00.598650 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:00.584157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.31.15.254:41555)
0809 16:37:00.584335 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:00.584346 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:00.584379 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:37:00.584497 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:00.584505 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:00.584563 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:00.584711 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:00.585485 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:00.586957 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:00.588000 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:00.591554 (+ 3554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:00.591690 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:00.594553 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:00.594557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:00.594558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:00.594571 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:00.596533 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:00.596905 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:00.596908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:00.596909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:00.597030 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:00.597219 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:00.597220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:00.597220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:00.597545 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:00.597705 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:00.598266 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 16:37:00.598408 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260809 16:37:00.603400 19180 catalog_manager.cc:1347] Generated new cluster ID: a4ad91a8a15c4b2d9f3f40da34f1c73f
I20260809 16:37:00.603463 19180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:37:00.637444 19180 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:37:00.638310 19180 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:00.647706 19180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Generated new TSK 0
I20260809 16:37:00.647984 19180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:37:01.254071 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:01.340794 19222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:01.340988 19222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:01.341029 19222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:01.342159 19222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:01.342202 19222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:01.342236 19222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:01.343674 19222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:01.343906 19222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:01.344115 19222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:01.348932 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.348985 19232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.349123 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:01.349866 19222 server_base.cc:1034] running on GCE node
I20260809 16:37:01.350060 19222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:01.350272 19222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:01.357570 19222 hybrid_clock.cc:648] HybridClock initialized: now 1786293421357540 us; error 39 us; skew 500 ppm
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:01.360113 19222 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:01.360562 19222 webserver.cc:460] Webserver started at http://127.31.15.193:37945/ using document root <none> and password file <none>
I20260809 16:37:01.360777 19222 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:01.360824 19222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:01.360956 19222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:01.375577 19222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "afe3805ebee2936f3e436afa5dd5f477"
server_key_iv: "7acf9ca5254eb0fd2d9e9713b82e44cd"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.376045 19222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "afe3805ebee2936f3e436afa5dd5f477"
server_key_iv: "7acf9ca5254eb0fd2d9e9713b82e44cd"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.377866 19222 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:01.397336 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.397532 19222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:01.397634 19222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "afe3805ebee2936f3e436afa5dd5f477"
server_key_iv: "7acf9ca5254eb0fd2d9e9713b82e44cd"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.397737 19222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:01.409559 19222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:01.411918 19222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:01.412077 19222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:01.412333 19222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:01.412743 19222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:01.412802 19222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.412838 19222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:01.412868 19222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.420563 19222 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:36443
I20260809 16:37:01.420655 19422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:36443 every 8 connection(s)
I20260809 16:37:01.420969 19222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:01.423736 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19222
I20260809 16:37:01.423856 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:37:01.424093 31807 external_mini_cluster.cc:1428] Setting key 85c9aa7494c8b945146940d077ffde5d
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
I20260809 16:37:01.436048 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.421809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:33969 (local address 127.31.15.254:41555)
0809 16:37:01.421970 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:01.421972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:01.422199 (+ 227us) server_negotiation.cc:407] Connection header received
0809 16:37:01.422847 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:01.422849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:01.422866 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:01.422962 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:01.423769 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.424556 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.425917 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.426196 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.426265 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.429459 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:01.429461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:01.429461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:01.429469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:01.431435 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.434853 (+ 3418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.434855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.434855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.434936 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.435245 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.435247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.435247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.435420 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:01.435467 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:01.435849 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 16:37:01.435943 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260809 16:37:01.436578 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.421733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.193:33969)
0809 16:37:01.422044 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:01.422091 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:01.422726 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:01.423068 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:01.423073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:01.423343 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:01.423649 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.423656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.424923 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.424924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.425799 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.425802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.426346 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.426347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.426502 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.427043 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:01.427054 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:01.429345 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:01.434482 (+ 5137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.434490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.434492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.434778 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.435011 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.435012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.435012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.435149 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.435518 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:01.435525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:01.435685 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 16:37:01.436180 (+ 495us) client_negotiation.cc:240] Negotiation successful
0809 16:37:01.436340 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":93,"threads_started":1}
I20260809 16:37:01.437173 19425 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:01.437278 19425 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:01.437493 19425 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:01.438064 19096 ts_manager.cc:194] Registered new tserver with Master: 12d0a2c5ca5e4ca1afbae9c6e801fa82 (127.31.15.193:36443)
I20260809 16:37:01.438669 19096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:33969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:37:01.456157 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:01.569896 19438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:01.570233 19438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:01.570317 19438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:01.572054 19438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:01.572165 19438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:01.572247 19438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:01.574434 19438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:01.574869 19438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:01.575192 19438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:01.581652 19451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.581977 19455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.582233 19448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:01.583225 19438 server_base.cc:1034] running on GCE node
I20260809 16:37:01.583455 19438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:01.583690 19438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:01.584851 19438 hybrid_clock.cc:648] HybridClock initialized: now 1786293421584840 us; error 45 us; skew 500 ppm
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:01.588217 19438 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:01.588672 19438 webserver.cc:460] Webserver started at http://127.31.15.194:39347/ using document root <none> and password file <none>
I20260809 16:37:01.588896 19438 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:01.588954 19438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:01.589072 19438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:01.602437 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "1957db0bcc7140ef917aecdbf994461e"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f85266217a5309a2c9f6d589e33dd51"
server_key_iv: "b46ca65c0e2385ec390399be4b9c6b4c"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.602857 19438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "1957db0bcc7140ef917aecdbf994461e"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f85266217a5309a2c9f6d589e33dd51"
server_key_iv: "b46ca65c0e2385ec390399be4b9c6b4c"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.612746 19438 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.000s sys 0.002s
I20260809 16:37:01.626180 19465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.626368 19438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:01.626472 19438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "1957db0bcc7140ef917aecdbf994461e"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f85266217a5309a2c9f6d589e33dd51"
server_key_iv: "b46ca65c0e2385ec390399be4b9c6b4c"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.626573 19438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:01.651835 19438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:01.654126 19438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:01.654335 19438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:01.654639 19438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:01.655038 19438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:01.655130 19438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.655211 19438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:01.655300 19438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.666069 19438 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:44915
I20260809 16:37:01.666551 19438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:37:01.668449 19633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:44915 every 8 connection(s)
I20260809 16:37:01.672597 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19438
I20260809 16:37:01.672765 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:37:01.672978 31807 external_mini_cluster.cc:1428] Setting key 75af0c483d8f1ab006b54772b419f77b
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:01.680362 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.669319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:48673 (local address 127.31.15.254:41555)
0809 16:37:01.669362 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:01.669364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:01.669831 (+ 467us) server_negotiation.cc:407] Connection header received
0809 16:37:01.670515 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:01.670516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:01.670530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:01.670560 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:01.671557 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.672368 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.674426 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.674754 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.674798 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.677357 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:01.677358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:01.677359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:01.677366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:01.678901 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.679396 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.679397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.679397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.679452 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.679711 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.679712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.679712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.679832 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:01.679887 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:01.680183 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 16:37:01.680264 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:37:01.680546 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.669024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.194:48673)
0809 16:37:01.669698 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:01.669744 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:01.670398 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:01.670745 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:01.670749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:01.671151 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:01.671447 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.671454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.672819 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.672820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.674321 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.674325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.674901 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.674902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.675043 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.675635 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:01.675645 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:01.677257 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:01.678993 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.679000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.679003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.679328 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.679513 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.679513 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.679514 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.679649 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.679890 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:01.679894 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:01.680049 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 16:37:01.680152 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:37:01.680299 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":78,"threads_started":1}
I20260809 16:37:01.681110 19634 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:01.681213 19634 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:01.681401 19634 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:01.681790 19096 ts_manager.cc:194] Registered new tserver with Master: 1957db0bcc7140ef917aecdbf994461e (127.31.15.194:44915)
I20260809 16:37:01.682286 19096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:48673
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:37:01.708405 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:01.821455 19646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:01.821810 19646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:01.821913 19646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:01.823606 19646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:01.823710 19646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:01.823794 19646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:01.825919 19646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:01.826402 19646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:01.826740 19646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:01.835306 19659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.835652 19656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:01.835904 19655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:01.836957 19646 server_base.cc:1034] running on GCE node
I20260809 16:37:01.837193 19646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:01.837474 19646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:01.838644 19646 hybrid_clock.cc:648] HybridClock initialized: now 1786293421838610 us; error 50 us; skew 500 ppm
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:01.841982 19646 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:01.842519 19646 webserver.cc:460] Webserver started at http://127.31.15.195:43271/ using document root <none> and password file <none>
I20260809 16:37:01.842763 19646 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:01.842828 19646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:01.843019 19646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:01.856858 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "073f77ebe1f44ae2b84b9e36a8519174"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f2e6e110e3f8a667728a9f2b26ca85b"
server_key_iv: "312630345bddceee9011594d2e787530"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.860698 19646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "073f77ebe1f44ae2b84b9e36a8519174"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f2e6e110e3f8a667728a9f2b26ca85b"
server_key_iv: "312630345bddceee9011594d2e787530"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.862428 19646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:01.883441 19671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.883684 19646 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260809 16:37:01.883795 19646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "073f77ebe1f44ae2b84b9e36a8519174"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f2e6e110e3f8a667728a9f2b26ca85b"
server_key_iv: "312630345bddceee9011594d2e787530"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:01.883903 19646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:01.914829 19646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:01.920312 19646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:01.920533 19646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:01.920809 19646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:01.921252 19646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:01.921347 19646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.921396 19646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:01.921415 19646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:01.929245 19646 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:38561
I20260809 16:37:01.929721 19646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:37:01.930413 19861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:38561 every 8 connection(s)
I20260809 16:37:01.934818 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19646
I20260809 16:37:01.934952 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:37:01.935149 31807 external_mini_cluster.cc:1428] Setting key 7504443b2415a04c5d0283d898468271
Aug 09 16:37:01 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293421, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:01.947664 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.930827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:48899 (local address 127.31.15.254:41555)
0809 16:37:01.930875 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:01.930877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:01.931000 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:37:01.931728 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:01.931729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:01.931742 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:01.931773 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:01.932612 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.933400 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.937626 (+ 4226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.937967 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.938015 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.941245 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:01.941247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:01.941248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:01.941256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:01.943001 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.943523 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.943524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.943524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.943595 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:01.943906 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:01.943907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:01.943907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:01.944065 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:01.944141 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:01.947464 (+ 3323us) server_negotiation.cc:299] Negotiation successful
0809 16:37:01.947560 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:37:01.947901 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:01.930530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.195:48899)
0809 16:37:01.930837 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:01.930882 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:01.931591 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:01.931973 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:01.931977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:01.932226 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:01.932500 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.932506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.933526 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.933527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.937518 (+ 3991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:01.937521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:01.938125 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:01.938126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:01.938271 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:01.939431 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:01.939442 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:01.941142 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:01.943092 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.943100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.943102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.943431 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.943673 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:01.943675 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:01.943676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:01.943825 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:01.944139 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:01.944144 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:01.944311 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 16:37:01.947409 (+ 3098us) client_negotiation.cc:240] Negotiation successful
0809 16:37:01.947590 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":92,"threads_started":1}
I20260809 16:37:01.948601 19862 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:01.948849 19862 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:01.949069 19862 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:01.949530 19093 ts_manager.cc:194] Registered new tserver with Master: 073f77ebe1f44ae2b84b9e36a8519174 (127.31.15.195:38561)
I20260809 16:37:01.950044 19093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:48899
I20260809 16:37:01.959697 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:01.960394 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19222
I20260809 16:37:02.000437 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19438
I20260809 16:37:02.008486 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19646
I20260809 16:37:02.025421 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19042
I20260809 16:37:02.039355 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:41555
--webserver_interface=127.31.15.254
--webserver_port=34975
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:41555
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:02.119724 19869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:02.119911 19869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:02.119931 19869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:02.120990 19869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:02.121023 19869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:02.121035 19869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:02.121047 19869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:02.121060 19869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:02.122725 19869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:41555
--ranger_config_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:41555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=34975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:02.122972 19869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:02.123188 19869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:37:02.127640 19869 server_base.cc:1034] running on GCE node
W20260809 16:37:02.127616 19875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.127691 19882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.127626 19877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:02.128049 19869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:02.128334 19869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:02.129487 19869 hybrid_clock.cc:648] HybridClock initialized: now 1786293422129473 us; error 27 us; skew 500 ppm
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:02.134531 19869 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:02.134958 19869 webserver.cc:460] Webserver started at http://127.31.15.254:34975/ using document root <none> and password file <none>
I20260809 16:37:02.135170 19869 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:02.135218 19869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:02.203048 19869 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.203886 19892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.204051 19869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:02.204160 19869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ca9b3a8d082a45508c0433ed614efd6f"
format_stamp: "Formatted at 2026-08-09 16:37:00 on dist-test-slave-21b9"
server_key: "4e2e8b9708a27dc543f01c4f817dc7e5"
server_key_iv: "13a5234823fd7996a1e41cb2c734c6c7"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:02.204602 19869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:02.222313 19869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:02.224164 19869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:02.224273 19869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:02.228314 19869 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:41555
I20260809 16:37:02.228368 19995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:41555 every 8 connection(s)
I20260809 16:37:02.228729 19869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:02.229945 19997 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:37:02.230605 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Bootstrap starting.
I20260809 16:37:02.235687 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19869
I20260809 16:37:02.235953 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:36443
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:41555
--webserver_port=37945
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:02.238291 19997 log.cc:824] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:02.239940 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:02.240183 19997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Bootstrap complete.
I20260809 16:37:02.242324 19997 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:02.242550 19997 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca9b3a8d082a45508c0433ed614efd6f, State: Initialized, Role: FOLLOWER
I20260809 16:37:02.242700 19997 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:02.242774 19997 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:02.242822 19997 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:02.242869 19997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:37:02.243875 19997 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:02.243980 19997 leader_election.cc:302] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca9b3a8d082a45508c0433ed614efd6f; no voters:
I20260809 16:37:02.244131 19997 leader_election.cc:288] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:37:02.244179 20006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:37:02.244313 20006 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [term 2 LEADER]: Becoming Leader. State: Replica: ca9b3a8d082a45508c0433ed614efd6f, State: Running, Role: LEADER
I20260809 16:37:02.244400 19997 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:02.244488 20006 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } }
I20260809 16:37:02.244827 20008 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ca9b3a8d082a45508c0433ed614efd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } } }
I20260809 16:37:02.244903 20008 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:02.245105 20009 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca9b3a8d082a45508c0433ed614efd6f. Latest consensus state: current_term: 2 leader_uuid: "ca9b3a8d082a45508c0433ed614efd6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca9b3a8d082a45508c0433ed614efd6f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 41555 } } }
I20260809 16:37:02.245213 20009 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:02.245326 20015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:02.245837 20015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:02.246024 20015 catalog_manager.cc:1259] Loaded cluster ID: a4ad91a8a15c4b2d9f3f40da34f1c73f
I20260809 16:37:02.246104 20015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:37:02.246158 19997 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 16:37:02.249639 20015 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:02.250195 20015 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Loaded TSK: 0
I20260809 16:37:02.250516 20015 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:37:02.352578 19999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:02.352962 19999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:02.353068 19999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:02.354851 19999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:02.354977 19999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:02.355067 19999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:02.357236 19999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:36443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=37945
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:02.357483 19999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:02.357798 19999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:02.364356 20059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.364459 20053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.364681 20049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:02.364778 19999 server_base.cc:1034] running on GCE node
I20260809 16:37:02.365010 19999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:02.365300 19999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:02.366480 19999 hybrid_clock.cc:648] HybridClock initialized: now 1786293422366427 us; error 64 us; skew 500 ppm
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:02.369757 19999 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:02.370256 19999 webserver.cc:460] Webserver started at http://127.31.15.193:37945/ using document root <none> and password file <none>
I20260809 16:37:02.370523 19999 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:02.370592 19999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:02.390057 19999 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.391309 20080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.391515 19999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.391611 19999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "afe3805ebee2936f3e436afa5dd5f477"
server_key_iv: "7acf9ca5254eb0fd2d9e9713b82e44cd"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:02.392068 19999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:02.412920 19999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:02.414904 19999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:02.415074 19999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:02.415339 19999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:02.415709 19999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:02.415758 19999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.415807 19999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:02.415832 19999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.423089 19999 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:36443
I20260809 16:37:02.423125 20274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:36443 every 8 connection(s)
I20260809 16:37:02.423586 19999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:02.424832 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 19999
I20260809 16:37:02.425141 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:44915
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:41555
--webserver_port=39347
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:02.436252 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.424332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.193:43895)
0809 16:37:02.424509 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:02.424550 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:02.425149 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:02.425765 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:02.425769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:02.425950 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:02.426217 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.426224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.427837 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.427838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.429069 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.429072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.430039 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.430040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.430170 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.430698 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:02.430707 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:02.432332 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:02.434425 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.434432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.434434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.434713 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.434982 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.434983 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.434983 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.435120 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.435614 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:02.435621 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:02.435765 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 16:37:02.435862 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 16:37:02.436020 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260809 16:37:02.437276 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.424461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:43895 (local address 127.31.15.254:41555)
0809 16:37:02.424704 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:02.424712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:02.424752 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:37:02.425431 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:02.425437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:02.425481 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:02.425610 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:02.426348 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.427717 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.429186 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.429797 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.429912 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.432450 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:02.432454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:02.432454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:02.432466 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:02.434333 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.434802 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.434805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.434806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.434912 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.435194 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.435195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.435196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.435508 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:02.435585 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:02.436949 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0809 16:37:02.437080 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":100,"threads_started":1}
I20260809 16:37:02.438143 20281 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:02.438246 20281 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.438427 20281 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.439334 20281 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:41555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:37:02.439406 20281 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.439468 20281 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:02.439900 20281 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.439976 20281 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.440326 20281 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:37:02.545833 20287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:02.546095 20287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:02.546139 20287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:02.547859 20287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:02.547923 20287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:02.547963 20287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:02.550155 20287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:44915
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=39347
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:02.550411 20287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:02.550701 20287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:02.556927 20300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.557298 20295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.557380 20296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:02.557616 20287 server_base.cc:1034] running on GCE node
I20260809 16:37:02.557832 20287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:02.558092 20287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:02.559263 20287 hybrid_clock.cc:648] HybridClock initialized: now 1786293422559239 us; error 45 us; skew 500 ppm
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:02.562559 20287 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:02.563123 20287 webserver.cc:460] Webserver started at http://127.31.15.194:39347/ using document root <none> and password file <none>
I20260809 16:37:02.563426 20287 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:02.563508 20287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:02.584355 20287 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.585453 20312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.585748 20287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.585872 20287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "1957db0bcc7140ef917aecdbf994461e"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f85266217a5309a2c9f6d589e33dd51"
server_key_iv: "b46ca65c0e2385ec390399be4b9c6b4c"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:02.586314 20287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:02.601341 20287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:02.603317 20287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:02.603472 20287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:02.603734 20287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:02.604116 20287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:02.604168 20287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.604199 20287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:02.604223 20287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.611485 20287 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:44915
I20260809 16:37:02.611974 20287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:37:02.612556 20499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:44915 every 8 connection(s)
I20260809 16:37:02.616135 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 20287
I20260809 16:37:02.616443 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.31.15.212:34927/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:38561
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:41555
--webserver_port=43271
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:02.622135 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.613045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40625 (local address 127.31.15.254:41555)
0809 16:37:02.613098 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:02.613100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:02.613110 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:37:02.613726 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:02.613728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:02.613746 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:02.613815 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:02.614492 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.615217 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.616973 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.617223 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.617258 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.619616 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:02.619617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:02.619618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:02.619626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:02.620883 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.621244 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.621245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.621246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.621303 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.621520 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.621521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.621521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.621635 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:02.621691 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:02.621950 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 16:37:02.622043 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:37:02.622272 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.612744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.194:40625)
0809 16:37:02.612909 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:02.612951 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:02.613561 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:02.613901 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:02.613905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:02.614123 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:02.614383 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.614390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.615347 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.615348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.616883 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.616885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.617347 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.617348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.617464 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.617950 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:02.617959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:02.619535 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:02.620960 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.620967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.620969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.621179 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.621357 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.621357 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.621358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.621464 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.621690 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:02.621697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:02.621830 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 16:37:02.621920 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:37:02.622059 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":73,"threads_started":1}
I20260809 16:37:02.622706 20500 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:02.622804 20500 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.623024 20500 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.623698 20500 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:41555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:37:02.623824 20500 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.623903 20500 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:02.624195 20500 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.624269 20500 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.624543 20500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:37:02.738790 20504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:02.739064 20504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:02.739101 20504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:02.740746 20504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:02.740811 20504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:02.740849 20504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:02.742906 20504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.31.15.212:34927/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:38561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=43271
--tserver_master_addrs=127.31.15.254:41555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:02.743176 20504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:02.743479 20504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:02.750157 20520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.750146 20517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:02.750556 20523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:02.751737 20504 server_base.cc:1034] running on GCE node
I20260809 16:37:02.751966 20504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:02.752251 20504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:02.753432 20504 hybrid_clock.cc:648] HybridClock initialized: now 1786293422753351 us; error 94 us; skew 500 ppm
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:02.756701 20504 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:02.757220 20504 webserver.cc:460] Webserver started at http://127.31.15.195:43271/ using document root <none> and password file <none>
I20260809 16:37:02.757495 20504 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:02.757573 20504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:02.776548 20504 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:02.777442 20538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.777624 20504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:02.777714 20504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "073f77ebe1f44ae2b84b9e36a8519174"
format_stamp: "Formatted at 2026-08-09 16:37:01 on dist-test-slave-21b9"
server_key: "5f2e6e110e3f8a667728a9f2b26ca85b"
server_key_iv: "312630345bddceee9011594d2e787530"
server_key_version: "kuduclusterkey@0"
I20260809 16:37:02.778096 20504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:02.796664 20504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:02.799546 20504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:02.799701 20504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:02.799945 20504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:02.800308 20504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:02.800354 20504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.800383 20504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:02.800406 20504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:02.807461 20504 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:38561
I20260809 16:37:02.807556 20717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:38561 every 8 connection(s)
I20260809 16:37:02.807863 20504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:37:02 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293422, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:02.815547 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 20504
I20260809 16:37:02.820220 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.808759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:51569 (local address 127.31.15.254:41555)
0809 16:37:02.808828 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:02.808831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:02.809013 (+ 182us) server_negotiation.cc:407] Connection header received
0809 16:37:02.809492 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:02.809493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:02.809506 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:02.809596 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:02.810163 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.810653 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.811728 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.811932 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.812002 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.816429 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:02.816431 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:02.816431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:02.816440 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:02.817862 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.818986 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.818987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.818988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.819067 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.819419 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.819421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.819421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.819565 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:02.819614 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:02.820032 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 16:37:02.820119 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:37:02.820895 20720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.808710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:41555 (local address 127.31.15.195:51569)
0809 16:37:02.808874 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:02.808917 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:02.809389 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:02.809656 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:02.809659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:02.809866 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:02.810069 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.810074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.810741 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.810742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.811645 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:02.811648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.812025 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.812027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:02.812141 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.812626 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:02.812635 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:02.816348 (+ 3713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:02.817943 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.817952 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.817955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.818269 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.819140 (+ 871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:02.819141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:02.819142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:02.819329 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:02.819665 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:02.819672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:02.819846 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 16:37:02.820470 (+ 624us) client_negotiation.cc:240] Negotiation successful
0809 16:37:02.820653 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260809 16:37:02.821374 20718 heartbeater.cc:347] Connected to a master server at 127.31.15.254:41555
I20260809 16:37:02.821471 20718 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.821671 20718 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.822301 20718 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:41555 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 16:37:02.822364 20718 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.822419 20718 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:02.822793 20718 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:02.822868 20718 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
W20260809 16:37:02.823167 20718 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:37:02.830806 20722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:02.816221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.31.15.254:41555)
0809 16:37:02.816466 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:02.816468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:02.816475 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:37:02.816502 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:02.816502 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:02.816514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:02.816578 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:02.818987 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.826121 (+ 7134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.826918 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:02.827135 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:02.827191 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:02.828441 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:02.828443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:02.828443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:02.828451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:02.829768 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.830092 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.830093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.830093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.830146 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:02.830310 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:02.830311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:02.830312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:02.830415 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:02.830593 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:02.830631 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:37:02.830709 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":76,"threads_started":1}
Aug 09 16:37:03 dist-test-slave-21b9 krb5kdc[18293](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786293423, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:03.440500 20281 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:03.440651 20281 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:03.441010 19925 ts_manager.cc:194] Registered new tserver with Master: 12d0a2c5ca5e4ca1afbae9c6e801fa82 (127.31.15.193:36443)
I20260809 16:37:03.441892 19925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:43895
I20260809 16:37:03.624711 20500 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:03.624871 20500 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:03.625290 19925 ts_manager.cc:194] Registered new tserver with Master: 1957db0bcc7140ef917aecdbf994461e (127.31.15.194:44915)
I20260809 16:37:03.625810 19925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:40625
Aug 09 16:37:03 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293423, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:03.823333 20718 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:03.823500 20718 heartbeater.cc:511] Master 127.31.15.254:41555 requested a full tablet report, sending...
I20260809 16:37:03.825400 19925 ts_manager.cc:194] Registered new tserver with Master: 073f77ebe1f44ae2b84b9e36a8519174 (127.31.15.195:38561)
I20260809 16:37:03.825927 19925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:51569
I20260809 16:37:03.832882 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:03.850831 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.839600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.31.15.254:41555)
0809 16:37:03.842128 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.842130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.842141 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:03.842168 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.842168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.842182 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.843120 (+ 938us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:03.843164 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.844264 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.845351 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.845632 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.846081 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.847405 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:03.847407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:03.847407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:03.847415 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:03.849265 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:03.849704 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:03.849705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:03.849705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:03.849771 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:03.850004 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:03.850004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:03.850005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:03.850131 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:03.850576 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.850624 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.850714 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2500,"thread_start_us":73,"threads_started":1}
I20260809 16:37:03.853890 19925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:37:03.859519 19925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:37:03.874235 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.866443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38561 (local address 127.0.0.1:56578)
0809 16:37:03.867925 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:03.867969 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:03.868039 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:03.868605 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:03.868610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:03.868613 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:03.868683 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.868685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.870265 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.870266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.872569 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.872572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.873915 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.873916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.874006 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.874022 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:37:03.874093 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:37:03.874146 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"thread_start_us":125,"threads_started":1}
I20260809 16:37:03.875351 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.866538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.31.15.195:38561)
0809 16:37:03.868233 (+ 1695us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.868238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.868250 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:37:03.868291 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.868293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.868311 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.868388 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:03.868774 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.870160 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.872673 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.873650 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.873798 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.873866 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.874773 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.875105 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1650,"thread_start_us":50,"threads_started":1}
I20260809 16:37:03.876360 20613 tablet_service.cc:1467] Processing CreateTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 (DEFAULT_TABLE table=test-table [id=2bd16a05f13548d483cb288e4ade52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:03.876801 20613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b779e6cd3af24ce3927e3ddcd06ded12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:03.882058 20757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.867024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:44915 (local address 127.0.0.1:55726)
0809 16:37:03.874654 (+ 7630us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:03.874664 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:03.874696 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:03.876173 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:03.876175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:03.876177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:03.876213 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.876214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.879415 (+ 3201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.879417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.880495 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.880498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.881786 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.881787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.881900 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.881905 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:37:03.881940 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 16:37:03.881981 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7619,"thread_start_us":128,"threads_started":1}
I20260809 16:37:03.882932 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.867361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.31.15.194:44915)
0809 16:37:03.875889 (+ 8528us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.875895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.875907 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:37:03.875942 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.875944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.875961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.876045 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:03.877899 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.879254 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.880602 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.881679 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.882233 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.882714 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.882745 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.882789 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8479,"thread_start_us":60,"threads_started":1}
I20260809 16:37:03.883685 20767 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Bootstrap starting.
I20260809 16:37:03.883965 20388 tablet_service.cc:1467] Processing CreateTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 (DEFAULT_TABLE table=test-table [id=2bd16a05f13548d483cb288e4ade52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:03.884322 20388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b779e6cd3af24ce3927e3ddcd06ded12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:03.885139 20767 tablet_bootstrap.cc:654] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:03.886174 20767 log.cc:824] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:03.888149 20771 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Bootstrap starting.
I20260809 16:37:03.888607 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.866745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36443 (local address 127.0.0.1:43604)
0809 16:37:03.871350 (+ 4605us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:03.871362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:03.871417 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:03.876497 (+ 5080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:03.876499 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:03.876501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:03.876562 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.876563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.877693 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.877694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.878766 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.878769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.888312 (+ 9543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.888314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.888419 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.888426 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:37:03.888485 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:37:03.888532 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4582,"thread_start_us":142,"threads_started":1}
I20260809 16:37:03.889087 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.866843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.31.15.193:36443)
0809 16:37:03.876343 (+ 9500us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.876347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.876358 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:03.876393 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.876395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.876411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.876492 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:03.876621 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.877610 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.886736 (+ 9126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.888202 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.888300 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.888367 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.888944 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.888992 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9459,"thread_start_us":53,"threads_started":1}
I20260809 16:37:03.889971 20166 tablet_service.cc:1467] Processing CreateTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 (DEFAULT_TABLE table=test-table [id=2bd16a05f13548d483cb288e4ade52dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:03.890380 20166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b779e6cd3af24ce3927e3ddcd06ded12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:03.897372 20771 tablet_bootstrap.cc:654] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:03.898422 20771 log.cc:824] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:03.899570 20767 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: No bootstrap required, opened a new log
I20260809 16:37:03.899566 20771 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: No bootstrap required, opened a new log
I20260809 16:37:03.899645 20771 ts_tablet_manager.cc:1397] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260809 16:37:03.899645 20767 ts_tablet_manager.cc:1397] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.000s
I20260809 16:37:03.903947 20771 raft_consensus.cc:348] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.904243 20771 raft_consensus.cc:374] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:03.904346 20771 raft_consensus.cc:729] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1957db0bcc7140ef917aecdbf994461e, State: Initialized, Role: FOLLOWER
I20260809 16:37:03.904574 20771 consensus_queue.cc:260] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.905030 20771 ts_tablet_manager.cc:1428] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260809 16:37:03.904963 20767 raft_consensus.cc:348] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.905167 20767 raft_consensus.cc:374] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:03.905194 20500 heartbeater.cc:503] Master 127.31.15.254:41555 was elected leader, sending a full tablet report...
I20260809 16:37:03.905253 20767 raft_consensus.cc:729] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073f77ebe1f44ae2b84b9e36a8519174, State: Initialized, Role: FOLLOWER
I20260809 16:37:03.905447 20767 consensus_queue.cc:260] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.905836 20767 ts_tablet_manager.cc:1428] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 16:37:03.905993 20718 heartbeater.cc:503] Master 127.31.15.254:41555 was elected leader, sending a full tablet report...
I20260809 16:37:03.915077 20772 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: Bootstrap starting.
I20260809 16:37:03.916690 20772 tablet_bootstrap.cc:654] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:03.917706 20772 log.cc:824] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:03.918903 20772 tablet_bootstrap.cc:492] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: No bootstrap required, opened a new log
I20260809 16:37:03.918988 20772 ts_tablet_manager.cc:1397] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 16:37:03.924584 20772 raft_consensus.cc:348] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.924835 20772 raft_consensus.cc:374] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:03.924926 20772 raft_consensus.cc:729] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d0a2c5ca5e4ca1afbae9c6e801fa82, State: Initialized, Role: FOLLOWER
I20260809 16:37:03.925138 20772 consensus_queue.cc:260] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.925602 20772 ts_tablet_manager.cc:1428] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260809 16:37:03.925778 20281 heartbeater.cc:503] Master 127.31.15.254:41555 was elected leader, sending a full tablet report...
I20260809 16:37:03.979974 20776 raft_consensus.cc:482] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:03.980223 20776 raft_consensus.cc:504] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.980786 20776 leader_election.cc:288] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073f77ebe1f44ae2b84b9e36a8519174 (127.31.15.195:38561), 12d0a2c5ca5e4ca1afbae9c6e801fa82 (127.31.15.193:36443)
I20260809 16:37:03.994486 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.982113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:52615 (local address 127.31.15.195:38561)
0809 16:37:03.982179 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.982181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.984014 (+ 1833us) server_negotiation.cc:407] Connection header received
0809 16:37:03.984125 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.984126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.984135 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.984164 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:03.984604 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.985301 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.990206 (+ 4905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.990908 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.990976 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.990988 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.994344 (+ 3356us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.994399 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:37:03.995137 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.981198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38561 (local address 127.31.15.194:52615)
0809 16:37:03.983933 (+ 2735us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:03.983946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:03.984048 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:03.984238 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:03.984238 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:03.984241 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:03.984501 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.984505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.985404 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.985405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.986970 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.986974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.994011 (+ 7037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.994013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.994221 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.994227 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:37:03.994822 (+ 595us) client_negotiation.cc:240] Negotiation successful
0809 16:37:03.994868 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2684,"thread_start_us":194,"threads_started":1}
I20260809 16:37:03.995455 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.982197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:44399 (local address 127.31.15.193:36443)
0809 16:37:03.982338 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:03.982341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:03.991565 (+ 9224us) server_negotiation.cc:407] Connection header received
0809 16:37:03.991675 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:03.991676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:03.991687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:03.991717 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:03.992142 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.992831 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.994400 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.994994 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.995045 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.995053 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:03.995349 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 16:37:03.995393 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260809 16:37:03.995757 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:03.981549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36443 (local address 127.31.15.194:44399)
0809 16:37:03.991481 (+ 9932us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:03.991495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:03.991597 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:03.991784 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:03.991785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:03.991786 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:03.992037 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.992039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.992934 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.992934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.994318 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:03.994321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:03.995111 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:03.995112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:03.995247 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:03.995251 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:37:03.995562 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 16:37:03.995602 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9878,"thread_start_us":130,"threads_started":1}
I20260809 16:37:03.996146 20648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b779e6cd3af24ce3927e3ddcd06ded12" candidate_uuid: "1957db0bcc7140ef917aecdbf994461e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073f77ebe1f44ae2b84b9e36a8519174" is_pre_election: true
I20260809 16:37:03.996337 20648 raft_consensus.cc:2454] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1957db0bcc7140ef917aecdbf994461e in term 0.
I20260809 16:37:03.996495 20195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b779e6cd3af24ce3927e3ddcd06ded12" candidate_uuid: "1957db0bcc7140ef917aecdbf994461e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" is_pre_election: true
I20260809 16:37:03.996727 20195 raft_consensus.cc:2454] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1957db0bcc7140ef917aecdbf994461e in term 0.
I20260809 16:37:03.996701 20314 leader_election.cc:302] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073f77ebe1f44ae2b84b9e36a8519174, 1957db0bcc7140ef917aecdbf994461e; no voters:
I20260809 16:37:03.996845 20776 raft_consensus.cc:2781] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:03.996896 20776 raft_consensus.cc:482] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:03.996922 20776 raft_consensus.cc:3037] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:03.998353 20776 raft_consensus.cc:504] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:03.998492 20776 leader_election.cc:288] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [CANDIDATE]: Term 1 election: Requested vote from peers 073f77ebe1f44ae2b84b9e36a8519174 (127.31.15.195:38561), 12d0a2c5ca5e4ca1afbae9c6e801fa82 (127.31.15.193:36443)
I20260809 16:37:03.998638 20648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b779e6cd3af24ce3927e3ddcd06ded12" candidate_uuid: "1957db0bcc7140ef917aecdbf994461e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073f77ebe1f44ae2b84b9e36a8519174"
I20260809 16:37:03.998659 20195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b779e6cd3af24ce3927e3ddcd06ded12" candidate_uuid: "1957db0bcc7140ef917aecdbf994461e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82"
I20260809 16:37:03.998711 20648 raft_consensus.cc:3037] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:03.998718 20195 raft_consensus.cc:3037] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:03.999972 20648 raft_consensus.cc:2454] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1957db0bcc7140ef917aecdbf994461e in term 1.
I20260809 16:37:04.000152 20314 leader_election.cc:302] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073f77ebe1f44ae2b84b9e36a8519174, 1957db0bcc7140ef917aecdbf994461e; no voters:
I20260809 16:37:04.000227 20195 raft_consensus.cc:2454] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1957db0bcc7140ef917aecdbf994461e in term 1.
I20260809 16:37:04.000239 20776 raft_consensus.cc:2781] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:04.000406 20776 raft_consensus.cc:686] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 1 LEADER]: Becoming Leader. State: Replica: 1957db0bcc7140ef917aecdbf994461e, State: Running, Role: LEADER
I20260809 16:37:04.000512 20776 consensus_queue.cc:237] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } }
I20260809 16:37:04.001358 19925 catalog_manager.cc:5495] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1957db0bcc7140ef917aecdbf994461e (127.31.15.194). New cstate: current_term: 1 leader_uuid: "1957db0bcc7140ef917aecdbf994461e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1957db0bcc7140ef917aecdbf994461e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 44915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 } health_report { overall_health: UNKNOWN } } }
I20260809 16:37:04.019603 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:04.016422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.31.15.194:44915)
0809 16:37:04.016493 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:04.016496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:04.016506 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:37:04.016548 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:04.016549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:04.016572 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:04.016651 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:04.016819 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:04.017600 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:04.018680 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:04.018994 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:04.019055 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:04.019208 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:04.019369 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:04.019451 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:04.019494 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 16:37:04.019517 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 16:37:04.023033 20195 raft_consensus.cc:1261] T b779e6cd3af24ce3927e3ddcd06ded12 P 12d0a2c5ca5e4ca1afbae9c6e801fa82 [term 1 FOLLOWER]: Refusing update from remote peer 1957db0bcc7140ef917aecdbf994461e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:04.023036 20648 raft_consensus.cc:1261] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 1 FOLLOWER]: Refusing update from remote peer 1957db0bcc7140ef917aecdbf994461e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:04.023408 20776 consensus_queue.cc:1035] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [LEADER]: Connected to new peer: Peer: permanent_uuid: "12d0a2c5ca5e4ca1afbae9c6e801fa82" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 36443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:04.023628 20784 consensus_queue.cc:1035] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [LEADER]: Connected to new peer: Peer: permanent_uuid: "073f77ebe1f44ae2b84b9e36a8519174" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:04.030230 20791 mvcc.cc:205] Tried to move back new op lower bound from 7316657864794464256 to 7316657864706727936. Current Snapshot: MvccSnapshot[applied={T|T < 7316657864794464256}]
I20260809 16:37:04.033267 19922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:04.033372 19922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:04.034313 19922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca9b3a8d082a45508c0433ed614efd6f: Sending DeleteTablet for 3 replicas of tablet b779e6cd3af24ce3927e3ddcd06ded12
I20260809 16:37:04.035517 20613 tablet_service.cc:1514] Processing DeleteTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:04 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:56578
I20260809 16:37:04.035967 20166 tablet_service.cc:1514] Processing DeleteTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:04 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:43604
I20260809 16:37:04.036337 20388 tablet_service.cc:1514] Processing DeleteTablet for tablet b779e6cd3af24ce3927e3ddcd06ded12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:04 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:55726
I20260809 16:37:04.037147 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19999
I20260809 16:37:04.042212 20792 mvcc.cc:205] Tried to move back new op lower bound from 7316657864794464256 to 7316657864706727936. Current Snapshot: MvccSnapshot[applied={T|T < 7316657864794464256}]
I20260809 16:37:04.049723 20802 tablet_replica.cc:330] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: stopping tablet replica
I20260809 16:37:04.049866 20802 raft_consensus.cc:2229] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:04.050019 20802 raft_consensus.cc:2258] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:04.051028 20802 ts_tablet_manager.cc:1905] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:04.053194 20802 ts_tablet_manager.cc:1918] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:04.053314 20802 log.cc:1196] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/b779e6cd3af24ce3927e3ddcd06ded12
I20260809 16:37:04.053808 20802 ts_tablet_manager.cc:1939] T b779e6cd3af24ce3927e3ddcd06ded12 P 073f77ebe1f44ae2b84b9e36a8519174: Deleting consensus metadata
I20260809 16:37:04.054419 19893 catalog_manager.cc:4841] TS 073f77ebe1f44ae2b84b9e36a8519174 (127.31.15.195:38561): tablet b779e6cd3af24ce3927e3ddcd06ded12 (table test-table [id=2bd16a05f13548d483cb288e4ade52dd]) successfully deleted
Aug 09 16:37:04 dist-test-slave-21b9 krb5kdc[18293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293423, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for HTTP/127.31.15.212@KRBTEST.COM
I20260809 16:37:04.068681 20804 tablet_replica.cc:330] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: stopping tablet replica
I20260809 16:37:04.068902 20804 raft_consensus.cc:2229] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:37:04.069078 20804 raft_consensus.cc:2258] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:04.069646 20804 ts_tablet_manager.cc:1905] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:37:04.071834 19893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:36443 (error 108)
I20260809 16:37:04.072216 20804 ts_tablet_manager.cc:1918] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:04.072324 20804 log.cc:1196] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/b779e6cd3af24ce3927e3ddcd06ded12
I20260809 16:37:04.072593 20804 ts_tablet_manager.cc:1939] T b779e6cd3af24ce3927e3ddcd06ded12 P 1957db0bcc7140ef917aecdbf994461e: Deleting consensus metadata
I20260809 16:37:04.072775 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 20287
I20260809 16:37:04.100663 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:04.100481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:36443 (local address 127.0.0.1:43618)
0809 16:37:04.100540 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:04.100611 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:36443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:04.100971 19893 catalog_manager.cc:4584] TS 12d0a2c5ca5e4ca1afbae9c6e801fa82 (127.31.15.193:36443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b779e6cd3af24ce3927e3ddcd06ded12: Network error: Client connection negotiation failed: client connection to 127.31.15.193:36443: connect: Connection refused (error 111)
I20260809 16:37:04.105386 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 20504
I20260809 16:37:04.107199 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:04.105508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:44915 (local address 127.0.0.1:55744)
0809 16:37:04.107108 (+ 1600us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:04.107143 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.194:44915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1575}
I20260809 16:37:04.116137 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 19869
I20260809 16:37:04.128942 31807 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 16:37:04.473044 31807 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 16:37:04.473146 31807 mini_ranger.cc:66] Stopping Ranger...
I20260809 16:37:04.816673 31807 mini_ranger.cc:68] Stopped Ranger
2026-08-09 16:37:04.816 UTC [18343] LOG: received smart shutdown request
2026-08-09 16:37:04.820 UTC [18343] LOG: background worker "logical replication launcher" (PID 18356) exited with exit code 1
2026-08-09 16:37:04.820 UTC [18349] LOG: shutting down
2026-08-09 16:37:04.945 UTC [18343] LOG: database system is shut down
2026-08-09T16:37:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46919 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:37:05 dist-test-slave-21b9 krb5kdc[20825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:37:05 dist-test-slave-21b9 krb5kdc[20825](info): set up 2 sockets
Aug 09 16:37:05 dist-test-slave-21b9 krb5kdc[20825](info): commencing operation
krb5kdc: starting...
W20260809 16:37:06.199218 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:37:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:37:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:37:06.312126 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:44229
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:06.391973 20842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:06.392163 20842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:06.392190 20842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:06.393306 20842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:06.393348 20842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:06.393368 20842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:06.393384 20842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:06.393399 20842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:06.394860 20842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:44229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:06.395118 20842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:06.395332 20842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:06.399827 20853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:06.399832 20851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:06.400032 20842 server_base.cc:1034] running on GCE node
W20260809 16:37:06.399835 20850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:06.400319 20842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:06.400604 20842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:06.401755 20842 hybrid_clock.cc:648] HybridClock initialized: now 1786293426401741 us; error 28 us; skew 500 ppm
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:06.404042 20842 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:06.404426 20842 webserver.cc:460] Webserver started at http://127.31.15.254:44907/ using document root <none> and password file <none>
I20260809 16:37:06.404625 20842 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:06.404673 20842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:06.404795 20842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:06.405755 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "2812f74ae07940c0bc308232f65811e6"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "20d4717dc216b6fd26d73a6f026f3cf3"
server_key_iv: "a84fccb723cc5ab8c02b035583bd95b9"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.406092 20842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "2812f74ae07940c0bc308232f65811e6"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "20d4717dc216b6fd26d73a6f026f3cf3"
server_key_iv: "a84fccb723cc5ab8c02b035583bd95b9"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.407255 20842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:37:06.407963 20863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:06.408108 20842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:06.408190 20842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "2812f74ae07940c0bc308232f65811e6"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "20d4717dc216b6fd26d73a6f026f3cf3"
server_key_iv: "a84fccb723cc5ab8c02b035583bd95b9"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.408255 20842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:06.428840 20842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:06.430474 20842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:06.430611 20842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:06.434360 20842 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:44229
I20260809 16:37:06.434398 20948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:44229 every 8 connection(s)
I20260809 16:37:06.434715 20842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:06.435602 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:06.436317 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 20842
I20260809 16:37:06.436424 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:37:06.436591 31807 external_mini_cluster.cc:1428] Setting key 0afe5b57e83c9cd70cfd1045284516d9
I20260809 16:37:06.437237 20949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.15.253@KRBTEST.COM; defaulting to no policy
I20260809 16:37:06.446453 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.437912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:44229 (local address 127.0.0.1:33622)
0809 16:37:06.438079 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.438117 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.438590 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.438843 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.438846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.439114 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.439262 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.439299 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.440291 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.440292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.440962 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.440964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.441408 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.441409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.441474 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.441929 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.441938 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.443490 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.445187 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.445194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.445196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.445390 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.445612 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.445613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.445613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.445688 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.446004 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.446012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.446077 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.446165 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.446369 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":62,"thread_start_us":97,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.253@KRBTEST.COM" created.
I20260809 16:37:06.446475 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.437919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33622 (local address 127.31.15.254:44229)
0809 16:37:06.438079 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.438089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.438203 (+ 114us) server_negotiation.cc:407] Connection header received
0809 16:37:06.438735 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.438737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.438753 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.438827 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.439392 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.440206 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.441038 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.441335 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.441429 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.443572 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.443579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.443580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.443586 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.445101 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.445474 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.445476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.445477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.445566 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.445730 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.445731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.445731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.445920 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.445979 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.446187 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.446274 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 16:37:06.447167 20949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.447499 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.447383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:49478)
0809 16:37:06.447426 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.447471 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 16:37:06.447604 20869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
I20260809 16:37:06.447991 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.447904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:49488)
0809 16:37:06.447945 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.447970 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:06.448086 20949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:40595: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
I20260809 16:37:06.461665 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:40595
--webserver_interface=127.31.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:06.497284 20949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } attempt: 1
I20260809 16:37:06.497809 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.497673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:49498)
0809 16:37:06.497736 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.497777 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:37:06.498181 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.498091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:49500)
0809 16:37:06.498134 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.498157 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:06.498281 20949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:40595: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
W20260809 16:37:06.542622 20964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:06.542819 20964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:06.542838 20964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:06.543917 20964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:06.543949 20964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:06.543962 20964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:06.543973 20964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:06.543983 20964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:06.545401 20964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:40595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:06.545594 20964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:06.545763 20964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:06.550382 20973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:06.550376 20975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:06.550397 20972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:06.550592 20964 server_base.cc:1034] running on GCE node
I20260809 16:37:06.550836 20964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:06.551111 20964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:06.552248 20964 hybrid_clock.cc:648] HybridClock initialized: now 1786293426552223 us; error 38 us; skew 500 ppm
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:06.554620 20964 init.cc:377] Logged in from keytab as kudu/127.31.15.253@KRBTEST.COM (short username kudu)
I20260809 16:37:06.555015 20964 webserver.cc:460] Webserver started at http://127.31.15.253:42193/ using document root <none> and password file <none>
I20260809 16:37:06.555224 20964 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:06.555285 20964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:06.555411 20964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:06.556434 20964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data/instance:
uuid: "8072341183df4e4b9fb6a5da7defd3f0"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "e2cdf716fe4be9b59aef3d4b980ae695"
server_key_iv: "a2f6625af7059b18d8c177600a019189"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.556751 20964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal/instance:
uuid: "8072341183df4e4b9fb6a5da7defd3f0"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "e2cdf716fe4be9b59aef3d4b980ae695"
server_key_iv: "a2f6625af7059b18d8c177600a019189"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.558027 20964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:37:06.558797 20982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:06.558988 20964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:37:06.559091 20964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "8072341183df4e4b9fb6a5da7defd3f0"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "e2cdf716fe4be9b59aef3d4b980ae695"
server_key_iv: "a2f6625af7059b18d8c177600a019189"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.559155 20964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:06.566452 20949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } attempt: 2
I20260809 16:37:06.566838 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.566705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:44024)
0809 16:37:06.566761 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.566802 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:37:06.567155 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.567087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:44040)
0809 16:37:06.567119 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.567141 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:37:06.567240 20949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:40595: Network error: Client connection negotiation failed: client connection to 127.31.15.253:40595: connect: Connection refused (error 111)
I20260809 16:37:06.574640 20964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:06.576680 20964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:06.576812 20964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:06.580659 20964 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.253:40595
I20260809 16:37:06.580693 21065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.253:40595 every 8 connection(s)
I20260809 16:37:06.581008 20964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
I20260809 16:37:06.581840 21067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:06.583343 21067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.586426 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 20964
I20260809 16:37:06.586526 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-1/wal/instance
I20260809 16:37:06.586695 31807 external_mini_cluster.cc:1428] Setting key c8e7dd3cd461c39fb0c51761b220ccbf
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:06.591634 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.583976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.31.15.254:44229)
0809 16:37:06.584047 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.584049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.584215 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:37:06.584655 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.584657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.584666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.584716 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.585259 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.585727 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.586712 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.586938 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.586993 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.588973 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.588975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.588975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.588982 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.590239 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.590718 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.590719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.590719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.590771 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.591028 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.591028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.591029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.591126 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.591173 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.591497 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.591572 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 16:37:06.591877 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.583949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:44229 (local address 127.0.0.1:54952)
0809 16:37:06.584104 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.584141 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.584584 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.584768 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.584771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.585005 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.585195 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.585199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.585815 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.585817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.586633 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.586636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.587031 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.587032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.587144 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.587589 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.587597 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.588885 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.590324 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.590334 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.590336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.590645 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.590826 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.590826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.590827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.590971 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.591181 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.591187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.591368 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.591486 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.591659 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260809 16:37:06.592460 21067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.15.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.252@KRBTEST.COM" created.
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:37:06.600764 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.592707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:44042)
0809 16:37:06.592754 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.592765 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.592808 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.592995 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.592996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.593137 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.593190 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.593191 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.593987 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.593988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.594734 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.594737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.595144 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.595145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.595219 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.595565 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.595569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.598187 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.599755 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.599756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.599756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.599955 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.600168 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.600169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.600169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.600221 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.600551 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.600551 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.600596 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.600641 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.600706 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:37:06.600879 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.592743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.31.15.253:40595)
0809 16:37:06.592871 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.592875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.592888 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:37:06.592906 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.592907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.592918 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.592977 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.593254 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.593903 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.594803 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.595070 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.595128 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.598266 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.598274 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.598274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.598279 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.599679 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.600050 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.600052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.600052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.600107 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.600300 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.600301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.600302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.600468 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.600578 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.600688 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.600764 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260809 16:37:06.601521 21067 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.601825 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.601728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58670)
0809 16:37:06.601766 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.601807 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:37:06.601958 20983 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
I20260809 16:37:06.602366 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.602289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58672)
0809 16:37:06.602327 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.602351 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:06.602465 21067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:45765: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
I20260809 16:37:06.611975 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:45765
--webserver_interface=127.31.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:06.651705 21067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } attempt: 1
I20260809 16:37:06.652179 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.652044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58688)
0809 16:37:06.652102 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.652146 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:37:06.652503 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.652424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58694)
0809 16:37:06.652463 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.652487 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:06.652601 21067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:45765: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
I20260809 16:37:06.658396 20949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } attempt: 3
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:37:06.665167 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.658725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44044 (local address 127.31.15.253:40595)
0809 16:37:06.658773 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.658775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.658784 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:37:06.658855 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.658856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.658864 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.658911 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.659256 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.659765 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.660720 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.660917 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.660964 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.662906 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.662907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.662907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.662914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.664182 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.664486 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.664487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.664487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.664537 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.664702 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.664702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.664703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.664809 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.664864 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.664999 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.665071 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:37:06.665174 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.658669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:44044)
0809 16:37:06.658723 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.658737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.658796 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.658930 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.658931 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.659115 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.659198 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.659200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.659830 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.659832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.660649 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.660652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.660976 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.660977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.661059 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.661491 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.661496 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.662823 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.664263 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.664264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.664265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.664426 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.664593 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.664593 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.664594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.664653 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.664862 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.664863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.664912 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.664971 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.665080 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:37:06.665514 20949 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.665833 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.665741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58704)
0809 16:37:06.665781 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.665806 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:37:06.666116 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.666059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58706)
0809 16:37:06.666084 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.666103 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 16:37:06.666194 20949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:45765: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
W20260809 16:37:06.694157 21080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:06.694344 21080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:06.694365 21080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:06.695458 21080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:06.695497 21080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:06.695509 21080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:06.695521 21080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:06.695535 21080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:06.697019 21080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:44229,127.31.15.253:40595,127.31.15.252:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:06.697207 21080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:06.697402 21080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:37:06.697355 20949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } attempt: 1
I20260809 16:37:06.697760 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.697642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58710)
0809 16:37:06.697695 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.697730 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:37:06.698127 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.698053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58716)
0809 16:37:06.698087 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.698111 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:37:06.698210 20949 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:45765: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
W20260809 16:37:06.701985 21094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:06.702127 21080 server_base.cc:1034] running on GCE node
W20260809 16:37:06.702056 21088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:06.702054 21091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:06.702419 21080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:06.702685 21080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:06.703832 21080 hybrid_clock.cc:648] HybridClock initialized: now 1786293426703818 us; error 27 us; skew 500 ppm
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:06.706127 21080 init.cc:377] Logged in from keytab as kudu/127.31.15.252@KRBTEST.COM (short username kudu)
I20260809 16:37:06.706586 21080 webserver.cc:460] Webserver started at http://127.31.15.252:40697/ using document root <none> and password file <none>
I20260809 16:37:06.706799 21080 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:06.706846 21080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:06.706969 21080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:06.708017 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data/instance:
uuid: "0bc536fed69649e3b2f1f64fdee4e71c"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "42f0c73ebd712b1071ddc36926f71fb4"
server_key_iv: "d7b92c400f9d335de23480cdfbe2f137"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.708335 21080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal/instance:
uuid: "0bc536fed69649e3b2f1f64fdee4e71c"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "42f0c73ebd712b1071ddc36926f71fb4"
server_key_iv: "d7b92c400f9d335de23480cdfbe2f137"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.709543 21080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:37:06.710153 21102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:06.710330 21080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:06.710402 21080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "0bc536fed69649e3b2f1f64fdee4e71c"
format_stamp: "Formatted at 2026-08-09 16:37:06 on dist-test-slave-21b9"
server_key: "42f0c73ebd712b1071ddc36926f71fb4"
server_key_iv: "d7b92c400f9d335de23480cdfbe2f137"
server_key_version: "encryptionkey@0"
I20260809 16:37:06.710469 21080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:06.716984 21080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:06.718526 21080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:06.718643 21080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:06.720742 21067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } attempt: 2
I20260809 16:37:06.721148 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.721000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58726)
0809 16:37:06.721064 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.721107 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:37:06.721519 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.721412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58736)
0809 16:37:06.721461 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.721493 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:37:06.721618 21067 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:45765: Network error: Client connection negotiation failed: client connection to 127.31.15.252:45765: connect: Connection refused (error 111)
I20260809 16:37:06.722615 21080 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.252:45765
I20260809 16:37:06.722667 21184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.252:45765 every 8 connection(s)
I20260809 16:37:06.722942 21080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
I20260809 16:37:06.723783 21185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:06.725260 21185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.726557 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 21080
I20260809 16:37:06.726661 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786293272397626-31807-0/minicluster-data/master-2/wal/instance
I20260809 16:37:06.726823 31807 external_mini_cluster.cc:1428] Setting key 68daed14975b013a5bf7e9430cdd359e
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:37:06.734007 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.725886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.31.15.254:44229)
0809 16:37:06.725943 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.725945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.726108 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:37:06.726586 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.726587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.726599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.726649 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.727436 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.727984 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.728952 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.729184 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.729232 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.731256 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.731258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.731258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.731307 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.732657 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.733117 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.733118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.733119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.733169 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.733412 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.733413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.733414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.733549 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.733620 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.733829 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.733910 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:37:06.734107 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.725841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:44229 (local address 127.0.0.1:54954)
0809 16:37:06.725988 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.726027 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.726483 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.726714 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.726718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.727010 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.727326 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.727333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.728083 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.728085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.728869 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.728871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.729256 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.729257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.729402 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.729846 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.729854 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.731177 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.732728 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.732735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.732737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.733051 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.733226 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.733227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.733227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.733354 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.733610 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.733613 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.733716 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.733791 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.733935 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":15,"thread_start_us":76,"threads_started":1}
I20260809 16:37:06.734588 21185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.735477 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.727161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.31.15.252:45765)
0809 16:37:06.727325 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.727329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.727450 (+ 121us) server_negotiation.cc:407] Connection header received
0809 16:37:06.727492 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.727495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.727515 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.727588 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.728093 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.728891 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.729773 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.730070 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.730120 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.732368 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.732377 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.732377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.732384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.734010 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.734563 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.734566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.734566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.734627 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.734897 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.734898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.734899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.735101 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.735289 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.735356 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.735413 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:06.740675 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.734812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:40595 (local address 127.0.0.1:44058)
0809 16:37:06.734847 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.734857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.734888 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.734991 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.734992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.735119 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.735170 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.735171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.735981 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.735982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.736518 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.736520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.736959 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.736960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.737035 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.737377 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.737381 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.738513 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.739824 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.739825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.739825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.739977 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.740146 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.740148 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.740148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.740202 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.740407 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.740408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.740449 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.740507 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.740589 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:37:06.740700 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.734866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.31.15.253:40595)
0809 16:37:06.734910 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.734912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.734917 (+ 5us) server_negotiation.cc:407] Connection header received
0809 16:37:06.734938 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.734938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.734945 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.734984 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.735230 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.735904 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.736608 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.736894 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.736949 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.738590 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.738591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.738591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.738598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.739754 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.740036 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.740037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.740037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.740095 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.740251 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.740252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.740252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.740354 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.740405 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.740545 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.740614 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:37:06.741398 21185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:37:06.742944 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.736256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.31.15.254:44229)
0809 16:37:06.736305 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.736306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.736312 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:37:06.736330 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.736330 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.736338 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.736380 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.736867 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.737461 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.738110 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.738323 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.738385 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.740401 (+ 2016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.740402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.740403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.740409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.741961 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.742320 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.742320 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.742321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.742367 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.742520 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.742520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.742520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.742608 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.742745 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.742780 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.742853 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:37:06.747803 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.741614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58756)
0809 16:37:06.741660 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.741670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.741723 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.741826 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.741826 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.741953 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.741986 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.741987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.742766 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.742767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.743440 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.743442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.743848 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.743849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.743887 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.744247 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.744252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.745495 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.746937 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.746938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.746938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.747093 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.747247 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.747247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.747248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.747327 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.747521 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.747522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.747557 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.747617 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.747708 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:37:06.747855 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.741645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.31.15.252:45765)
0809 16:37:06.741676 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.741677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.741708 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:37:06.741775 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.741776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.741783 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.741821 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.742049 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.742690 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.743504 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.743782 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.743849 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.745573 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.745575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.745575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.745580 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.746864 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.747150 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.747151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.747151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.747196 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.747381 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.747382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.747382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.747470 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.747522 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.747643 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.747757 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":13}
I20260809 16:37:06.749433 21185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: Bootstrap starting.
I20260809 16:37:06.750341 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.743606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.31.15.253:40595)
0809 16:37:06.743640 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.743641 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.743647 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:37:06.743661 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.743661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.743668 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.743700 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.744218 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.744843 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.745706 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.746021 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.746104 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.748203 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.748204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.748205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.748211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.749422 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.749756 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.749757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.749757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.749798 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.749966 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.749966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.749967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.750049 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.750179 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.750225 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.750282 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:37:06.750491 21185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:06.750802 21185 log.cc:824] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:06.751696 21185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: No bootstrap required, opened a new log
I20260809 16:37:06.751865 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:37:06.753589 21185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.753729 21185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:06.753764 21185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc536fed69649e3b2f1f64fdee4e71c, State: Initialized, Role: FOLLOWER
I20260809 16:37:06.753906 21185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.754208 21185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:06.754268 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.754437 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:37:06.756170 21230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:37:06.756269 21230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:37:06.757102 21080 master_runner.cc:186] Error getting master registration for 127.31.15.254:44229: OK, instance_id { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" instance_seqno: 1786293426433373 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:37:06.759963 21080 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:37:06.760344 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.753165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.31.15.254:44229)
0809 16:37:06.753200 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.753202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.753312 (+ 110us) server_negotiation.cc:407] Connection header received
0809 16:37:06.753371 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.753372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.753382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.753414 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.754165 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.754775 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.755687 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.755940 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.755983 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.757537 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.757539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.757539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.757546 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.759038 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.759420 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.759422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.759422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.759477 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.759674 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.759675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.759676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.759796 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.759880 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.760187 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.760256 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:37:06.761304 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.753123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.31.15.253:40595)
0809 16:37:06.753162 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.753164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.753433 (+ 269us) server_negotiation.cc:407] Connection header received
0809 16:37:06.753507 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.753508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.753518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.753574 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.754176 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.754823 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.755816 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.756086 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.756145 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.757720 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.757721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.757721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.757727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.760132 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.760472 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.760473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.760473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.760526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.760704 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.760705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.760705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.760806 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.760885 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.761138 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.761213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:37:06.762444 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.753410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58770 (local address 127.31.15.252:45765)
0809 16:37:06.753440 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.753441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.755770 (+ 2329us) server_negotiation.cc:407] Connection header received
0809 16:37:06.755856 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.755856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.755865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.755890 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.756547 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.757211 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.758103 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.758361 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.758400 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.759658 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.759660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.759660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.759667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.761353 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.761715 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.761716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.761716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.761768 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.761974 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.761975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.761975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.762104 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.762170 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.762312 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.762388 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 16:37:06.773392 20949 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } attempt: 2
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:37:06.780478 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.773771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.31.15.252:45765)
0809 16:37:06.773807 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.773809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.773816 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:37:06.773834 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.773835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.773843 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.773886 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.774230 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.774684 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.775664 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.775838 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.775903 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.777775 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.777776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.777776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.777783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.779485 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.779837 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.779838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.779838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.779894 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.780066 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.780067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.780067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.780160 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.780208 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.780359 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.780418 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:37:06.780503 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.773688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58780)
0809 16:37:06.773732 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.773743 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.773784 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.773928 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.773929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.774109 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.774153 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.774155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.774759 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.774760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.775591 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.775593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.775903 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.775904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.775958 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.776341 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.776346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.777681 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.779557 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.779559 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.779559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.779771 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.779948 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.779949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.779949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.780008 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.780209 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.780210 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.780254 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.780315 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.780404 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:37:06.781827 20949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: Bootstrap starting.
I20260809 16:37:06.782469 20949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:06.782747 20949 log.cc:824] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:06.783545 20949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: No bootstrap required, opened a new log
I20260809 16:37:06.784979 20949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.785094 20949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:06.785132 20949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2812f74ae07940c0bc308232f65811e6, State: Initialized, Role: FOLLOWER
I20260809 16:37:06.785255 20949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.785521 21233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.785570 20949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:06.785622 21233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:37:06.787593 21252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:37:06.787658 21252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:37:06.788268 20842 master_runner.cc:186] Error getting master registration for 127.31.15.253:40595: OK, instance_id { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" instance_seqno: 1786293426579605 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:37:06.790307 20842 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:37:06.812755 21067 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } attempt: 3
Aug 09 16:37:06 dist-test-slave-21b9 krb5kdc[20825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293426, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:37:06.819195 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.813105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58786 (local address 127.31.15.252:45765)
0809 16:37:06.813142 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:06.813144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:06.813167 (+ 23us) server_negotiation.cc:407] Connection header received
0809 16:37:06.813190 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:06.813191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:06.813200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:06.813243 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:06.813552 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.814074 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.814732 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.814915 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.814967 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.816715 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:06.816716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:06.816716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:06.816723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:06.818022 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.818416 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.818417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.818418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.818479 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:06.818681 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:06.818682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:06.818682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:06.818826 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:06.818885 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:06.819033 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:37:06.819100 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:37:06.819196 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:06.813049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:45765 (local address 127.0.0.1:58786)
0809 16:37:06.813100 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:06.813111 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:06.813149 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:06.813287 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:06.813288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:06.813438 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:06.813480 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.813481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.814136 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.814137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.814656 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:06.814659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:06.814969 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:06.814970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:06.815032 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:06.815478 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:06.815483 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:06.816637 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:06.818130 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.818131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.818131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.818341 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.818547 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:06.818548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:06.818548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:06.818615 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:06.818891 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:06.818892 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:06.818940 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 16:37:06.818999 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:37:06.819100 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:37:06.820686 21067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: Bootstrap starting.
I20260809 16:37:06.821311 21067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:06.821594 21067 log.cc:824] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:06.822307 21067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: No bootstrap required, opened a new log
I20260809 16:37:06.823807 21067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.823916 21067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:06.823948 21067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8072341183df4e4b9fb6a5da7defd3f0, State: Initialized, Role: FOLLOWER
I20260809 16:37:06.824113 21067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.824369 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.824430 21067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:06.824509 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 16:37:06.826220 21274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:37:06.826300 21274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 16:37:06.827368 20989 tablet.cc:2369] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:06.828382 20964 master_runner.cc:263] No leader master found from master 8072341183df4e4b9fb6a5da7defd3f0
I20260809 16:37:06.828430 20964 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 16:37:06.931375 20871 tablet.cc:2369] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:06.965027 21213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:06.965152 21213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.965399 21213 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2812f74ae07940c0bc308232f65811e6 (127.31.15.254:44229), 8072341183df4e4b9fb6a5da7defd3f0 (127.31.15.253:40595)
I20260809 16:37:06.965637 20903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2812f74ae07940c0bc308232f65811e6" is_pre_election: true
I20260809 16:37:06.965687 21019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8072341183df4e4b9fb6a5da7defd3f0" is_pre_election: true
I20260809 16:37:06.965773 20903 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc536fed69649e3b2f1f64fdee4e71c in term 0.
I20260809 16:37:06.965811 21019 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc536fed69649e3b2f1f64fdee4e71c in term 0.
I20260809 16:37:06.965971 21105 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc536fed69649e3b2f1f64fdee4e71c, 2812f74ae07940c0bc308232f65811e6; no voters:
I20260809 16:37:06.966122 21213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:06.966194 21213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:06.966225 21213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:06.966928 21213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.967043 21213 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [CANDIDATE]: Term 1 election: Requested vote from peers 2812f74ae07940c0bc308232f65811e6 (127.31.15.254:44229), 8072341183df4e4b9fb6a5da7defd3f0 (127.31.15.253:40595)
I20260809 16:37:06.967160 20903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2812f74ae07940c0bc308232f65811e6"
I20260809 16:37:06.967196 21019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8072341183df4e4b9fb6a5da7defd3f0"
I20260809 16:37:06.967252 20903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:06.967263 21019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:06.968092 21019 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc536fed69649e3b2f1f64fdee4e71c in term 1.
I20260809 16:37:06.968092 20903 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc536fed69649e3b2f1f64fdee4e71c in term 1.
I20260809 16:37:06.968255 21105 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0bc536fed69649e3b2f1f64fdee4e71c, 2812f74ae07940c0bc308232f65811e6, 8072341183df4e4b9fb6a5da7defd3f0; no voters:
I20260809 16:37:06.968349 21213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:06.968508 21213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [term 1 LEADER]: Becoming Leader. State: Replica: 0bc536fed69649e3b2f1f64fdee4e71c, State: Running, Role: LEADER
I20260809 16:37:06.968600 21213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } }
I20260809 16:37:06.968971 21278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc536fed69649e3b2f1f64fdee4e71c. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.969067 21278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: This master's current role is: LEADER
W20260809 16:37:06.969174 21110 tablet.cc:2369] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:06.969280 21283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:06.969445 21283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:06.970688 21019 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [term 1 FOLLOWER]: Refusing update from remote peer 0bc536fed69649e3b2f1f64fdee4e71c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:06.970688 20903 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [term 1 FOLLOWER]: Refusing update from remote peer 0bc536fed69649e3b2f1f64fdee4e71c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:06.970996 21278 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:06.971117 21213 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [LEADER]: Connected to new peer: Peer: permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:06.972251 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc536fed69649e3b2f1f64fdee4e71c. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.972251 21233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc536fed69649e3b2f1f64fdee4e71c. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.972340 21233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:37:06.972340 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:37:06.973150 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.973232 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:06.973318 21213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.973359 21213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:06.973397 21285 mvcc.cc:205] Tried to move back new op lower bound from 7316657876869869568 to 7316657876863975424. Current Snapshot: MvccSnapshot[applied={T|T < 7316657876869869568 or (T in {7316657876869869568})}]
I20260809 16:37:06.973598 21283 catalog_manager.cc:1347] Generated new cluster ID: 0d70bb07ff0a4a65ab401f7e15721957
I20260809 16:37:06.973644 21283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:37:06.973865 21233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.973865 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2812f74ae07940c0bc308232f65811e6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44229 } } peers { permanent_uuid: "8072341183df4e4b9fb6a5da7defd3f0" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 40595 } } peers { permanent_uuid: "0bc536fed69649e3b2f1f64fdee4e71c" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 45765 } } }
I20260809 16:37:06.973942 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:37:06.973942 21233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:37:06.973955 21287 mvcc.cc:205] Tried to move back new op lower bound from 7316657876869869568 to 7316657876863975424. Current Snapshot: MvccSnapshot[applied={T|T < 7316657876869869568}]
I20260809 16:37:06.974043 21286 mvcc.cc:205] Tried to move back new op lower bound from 7316657876869869568 to 7316657876863975424. Current Snapshot: MvccSnapshot[applied={T|T < 7316657876869869568}]
I20260809 16:37:06.997018 21283 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:37:06.997623 21283 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:07.004294 21283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bc536fed69649e3b2f1f64fdee4e71c: Generated new TSK 0
I20260809 16:37:07.004539 21283 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:37:07.788100 21252 catalog_manager.cc:1259] Loaded cluster ID: 0d70bb07ff0a4a65ab401f7e15721957
I20260809 16:37:07.788187 21252 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: loading cluster ID for follower catalog manager: success
I20260809 16:37:07.789011 21252 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: acquiring CA information for follower catalog manager: success
I20260809 16:37:07.789337 21252 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2812f74ae07940c0bc308232f65811e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:37:07.826756 21274 catalog_manager.cc:1259] Loaded cluster ID: 0d70bb07ff0a4a65ab401f7e15721957
I20260809 16:37:07.826826 21274 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: loading cluster ID for follower catalog manager: success
I20260809 16:37:07.827658 21274 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: acquiring CA information for follower catalog manager: success
I20260809 16:37:07.827972 21274 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8072341183df4e4b9fb6a5da7defd3f0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:37:08.060815 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 20842
I20260809 16:37:08.080997 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 20964
I20260809 16:37:08.092202 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 21080
2026-08-09T16:37:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3062 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:37:08 dist-test-slave-21b9 krb5kdc[21307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:37:08 dist-test-slave-21b9 krb5kdc[21307](info): set up 2 sockets
Aug 09 16:37:08 dist-test-slave-21b9 krb5kdc[21307](info): commencing operation
krb5kdc: starting...
W20260809 16:37:09.229532 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:37:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:37:09.343127 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36151
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:45585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:36151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:09.423213 21324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:09.423448 21324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:09.423481 21324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:09.424602 21324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:09.424643 21324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:09.424661 21324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:09.424676 21324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:09.424691 21324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:09.426172 21324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:45585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:36151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:09.426420 21324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:09.426628 21324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:09.431121 21340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:09.431248 21324 server_base.cc:1034] running on GCE node
W20260809 16:37:09.431129 21336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:09.431154 21337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:09.431605 21324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:09.431880 21324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:09.433017 21324 hybrid_clock.cc:648] HybridClock initialized: now 1786293429433000 us; error 31 us; skew 500 ppm
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293429, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:09.435377 21324 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:09.435801 21324 webserver.cc:460] Webserver started at http://127.31.15.254:45029/ using document root <none> and password file <none>
I20260809 16:37:09.436023 21324 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:09.436071 21324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:09.436197 21324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:09.437201 21324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "cfd05b04c0ea4b10b76979680d7fa8a6"
format_stamp: "Formatted at 2026-08-09 16:37:09 on dist-test-slave-21b9"
server_key: "6b2970b079fa554bd45df54dc61fbb92"
server_key_iv: "f063d56920c2c9cc43a445d10d449f7e"
server_key_version: "encryptionkey@0"
I20260809 16:37:09.437529 21324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "cfd05b04c0ea4b10b76979680d7fa8a6"
format_stamp: "Formatted at 2026-08-09 16:37:09 on dist-test-slave-21b9"
server_key: "6b2970b079fa554bd45df54dc61fbb92"
server_key_iv: "f063d56920c2c9cc43a445d10d449f7e"
server_key_version: "encryptionkey@0"
I20260809 16:37:09.438767 21324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:37:09.439514 21351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:09.439677 21324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:09.439761 21324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "cfd05b04c0ea4b10b76979680d7fa8a6"
format_stamp: "Formatted at 2026-08-09 16:37:09 on dist-test-slave-21b9"
server_key: "6b2970b079fa554bd45df54dc61fbb92"
server_key_iv: "f063d56920c2c9cc43a445d10d449f7e"
server_key_version: "encryptionkey@0"
I20260809 16:37:09.439821 21324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:09.454888 21324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:09.456951 21324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:09.457085 21324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:09.461124 21324 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:36151
I20260809 16:37:09.461201 21426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:36151 every 8 connection(s)
I20260809 16:37:09.461521 21324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:09.462375 21428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:09.464608 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6: Bootstrap starting.
I20260809 16:37:09.465162 21428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:09.465444 21428 log.cc:824] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:09.466110 21428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6: No bootstrap required, opened a new log
I20260809 16:37:09.467563 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 21324
I20260809 16:37:09.467661 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:37:09.467823 31807 external_mini_cluster.cc:1428] Setting key 41035a9a53d07f61fe77df67ec3591b8
I20260809 16:37:09.467739 21428 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } }
I20260809 16:37:09.467867 21428 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:09.467912 21428 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd05b04c0ea4b10b76979680d7fa8a6, State: Initialized, Role: FOLLOWER
I20260809 16:37:09.468058 21428 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } }
I20260809 16:37:09.468145 21428 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:09.468222 21428 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:09.468283 21428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:09.469262 21428 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } }
I20260809 16:37:09.469350 21428 leader_election.cc:302] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfd05b04c0ea4b10b76979680d7fa8a6; no voters:
I20260809 16:37:09.469491 21428 leader_election.cc:288] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:37:09.469527 21436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:09.469676 21436 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [term 1 LEADER]: Becoming Leader. State: Replica: cfd05b04c0ea4b10b76979680d7fa8a6, State: Running, Role: LEADER
I20260809 16:37:09.469749 21428 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:09.469812 21436 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } }
I20260809 16:37:09.470289 21437 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } } }
I20260809 16:37:09.470386 21437 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:09.470535 21438 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfd05b04c0ea4b10b76979680d7fa8a6. Latest consensus state: current_term: 1 leader_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfd05b04c0ea4b10b76979680d7fa8a6" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36151 } } }
I20260809 16:37:09.470588 21438 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:09.470837 21446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:09.471377 21446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:09.472890 21446 catalog_manager.cc:1347] Generated new cluster ID: faa24db3b3ec458eaec7103758ce583e
I20260809 16:37:09.472940 21446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:09.477790 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:09.468223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.31.15.254:36151)
0809 16:37:09.468593 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:09.468603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:09.468635 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:37:09.468761 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:09.468768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:09.468825 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:09.468972 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:09.469647 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:09.470985 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:09.472336 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:09.472752 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:09.472856 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:09.474910 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:09.474913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:09.474914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:09.474923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:09.476422 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:09.476730 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:09.476733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:09.476733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:09.476837 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:09.477010 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:09.477010 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:09.477011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:09.477222 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:09.477369 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:09.477486 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 16:37:09.477616 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":91,"threads_started":1}
I20260809 16:37:09.478673 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:37:09.490288 21446 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:37:09.490978 21446 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:09.496737 21446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfd05b04c0ea4b10b76979680d7fa8a6: Generated new TSK 0
I20260809 16:37:09.496897 21446 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:37:09.532284 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 21324
2026-08-09T16:37:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1446 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21465](info): set up 2 sockets
Aug 09 16:37:09 dist-test-slave-21b9 krb5kdc[21465](info): commencing operation
krb5kdc: starting...
W20260809 16:37:10.685075 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:37:10 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:37:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:37:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:37:10.793396 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:10.871848 21484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:10.872035 21484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:10.872062 21484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:10.873140 21484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:10.873178 21484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:10.873196 21484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:10.873211 21484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:10.873227 21484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:10.874634 21484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:10.874859 21484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:10.875048 21484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:10.879537 21491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:10.879524 21489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:10.879524 21495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:10.879549 21484 server_base.cc:1034] running on GCE node
I20260809 16:37:10.879899 21484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:10.880117 21484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:10.881261 21484 hybrid_clock.cc:648] HybridClock initialized: now 1786293430881249 us; error 30 us; skew 500 ppm
Aug 09 16:37:10 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293430, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:10.883530 21484 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:10.883879 21484 webserver.cc:460] Webserver started at http://127.31.15.254:41641/ using document root <none> and password file <none>
I20260809 16:37:10.884039 21484 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:10.884076 21484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:10.884157 21484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:10.885053 21484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4"
format_stamp: "Formatted at 2026-08-09 16:37:10 on dist-test-slave-21b9"
server_key: "5c9887e03db8933bd26339ec41c81280"
server_key_iv: "fe714281f6eb5b7e1bb540f107fd7882"
server_key_version: "encryptionkey@0"
I20260809 16:37:10.885322 21484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4"
format_stamp: "Formatted at 2026-08-09 16:37:10 on dist-test-slave-21b9"
server_key: "5c9887e03db8933bd26339ec41c81280"
server_key_iv: "fe714281f6eb5b7e1bb540f107fd7882"
server_key_version: "encryptionkey@0"
I20260809 16:37:10.886452 21484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:37:10.887072 21509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:10.887228 21484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:10.887332 21484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4"
format_stamp: "Formatted at 2026-08-09 16:37:10 on dist-test-slave-21b9"
server_key: "5c9887e03db8933bd26339ec41c81280"
server_key_iv: "fe714281f6eb5b7e1bb540f107fd7882"
server_key_version: "encryptionkey@0"
I20260809 16:37:10.887387 21484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:10.907473 21484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:10.909034 21484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:10.909154 21484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:10.912838 21484 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38255
I20260809 16:37:10.912873 21594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38255 every 8 connection(s)
I20260809 16:37:10.913167 21484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:10.914062 21596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:10.916183 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap starting.
I20260809 16:37:10.916726 21596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:10.916985 21596 log.cc:824] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:10.917431 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 21484
I20260809 16:37:10.917533 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:37:10.917701 31807 external_mini_cluster.cc:1428] Setting key 76b2adca1792b911f84913c66be238aa
I20260809 16:37:10.917697 21596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: No bootstrap required, opened a new log
I20260809 16:37:10.919798 21596 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:10.919965 21596 raft_consensus.cc:374] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:10.920004 21596 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Initialized, Role: FOLLOWER
I20260809 16:37:10.920133 21596 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:10.920205 21596 raft_consensus.cc:388] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:10.920240 21596 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:10.920284 21596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:10.920970 21596 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:10.921075 21596 leader_election.cc:302] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ded5a6fa9a8d4cbd9bc413bebd4d9cf4; no voters:
I20260809 16:37:10.921219 21596 leader_election.cc:288] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:37:10.921265 21603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:10.921454 21596 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:10.921499 21603 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 LEADER]: Becoming Leader. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Running, Role: LEADER
I20260809 16:37:10.921586 21603 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:10.921891 21603 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded5a6fa9a8d4cbd9bc413bebd4d9cf4. Latest consensus state: current_term: 1 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:10.921950 21603 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:10.922139 21615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:10.922777 21604 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:10.922852 21604 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:10.923173 21615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:10.924755 21615 catalog_manager.cc:1347] Generated new cluster ID: a4b8f126917a410395324c08349b2590
I20260809 16:37:10.924811 21615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:37:10 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:10.928427 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:10.918141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55698 (local address 127.31.15.254:38255)
0809 16:37:10.918325 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:10.918335 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:10.918369 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:37:10.919001 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:10.919011 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:10.919071 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:10.919213 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:10.919773 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:10.921362 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:10.922423 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:10.923003 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:10.923105 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:10.925547 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:10.925550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:10.925551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:10.925562 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:10.927047 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:10.927382 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:10.927385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:10.927385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:10.927476 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:10.927655 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:10.927656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:10.927656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:10.927871 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:10.928011 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:10.928140 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 16:37:10.928267 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260809 16:37:10.941926 21615 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:37:10.942466 21615 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:10.947146 21615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Generated new TSK 0
I20260809 16:37:10.947340 21615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:37:10.975561 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:11.057523 21636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:11.057729 21636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:11.057754 21636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:11.058885 21636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:11.058931 21636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:11.058957 21636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:11.060462 21636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:11.060706 21636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:11.060931 21636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:11.065897 21649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.065894 21645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.065958 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:11.065935 21636 server_base.cc:1034] running on GCE node
I20260809 16:37:11.066298 21636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:11.066541 21636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:11.067682 21636 hybrid_clock.cc:648] HybridClock initialized: now 1786293431067663 us; error 33 us; skew 500 ppm
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:11.070169 21636 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:11.070593 21636 webserver.cc:460] Webserver started at http://127.31.15.193:38761/ using document root <none> and password file <none>
I20260809 16:37:11.070804 21636 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:11.070852 21636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:11.070981 21636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:11.071998 21636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "849f607764fa4aeea15e11dafbf38ef1"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "084bb4ab40a70af56fb822b571cbb159"
server_key_iv: "84bde4f3e343fbfabf502a9469308db7"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.072324 21636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "849f607764fa4aeea15e11dafbf38ef1"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "084bb4ab40a70af56fb822b571cbb159"
server_key_iv: "84bde4f3e343fbfabf502a9469308db7"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.073567 21636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:37:11.074278 21661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.074455 21636 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:11.074541 21636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "849f607764fa4aeea15e11dafbf38ef1"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "084bb4ab40a70af56fb822b571cbb159"
server_key_iv: "84bde4f3e343fbfabf502a9469308db7"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.074618 21636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:11.113481 21636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:11.115378 21636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:11.115523 21636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:11.115736 21636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:11.116040 21636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:11.116077 21636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.116107 21636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:11.116130 21636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.121584 21636 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:43903
I20260809 16:37:11.121647 21844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:43903 every 8 connection(s)
I20260809 16:37:11.121920 21636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:11.130312 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 21636
I20260809 16:37:11.130422 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:37:11.130628 31807 external_mini_cluster.cc:1428] Setting key 22619e816a8d20df4592089f5be19b73
I20260809 16:37:11.130725 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.122620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:33183 (local address 127.31.15.254:38255)
0809 16:37:11.122673 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.122675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.122871 (+ 196us) server_negotiation.cc:407] Connection header received
0809 16:37:11.123484 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.123486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.123498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.123548 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:11.124442 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.124952 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.125906 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.126123 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.126180 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.128142 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:11.128144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:11.128144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:11.128152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:11.129426 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.129781 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.129782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.129783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.129833 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.130048 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.130049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.130049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.130168 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:11.130238 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.130546 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.130635 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:37:11.130856 21850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.122596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:33183)
0809 16:37:11.122748 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.122789 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.123370 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.123620 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.123624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.124165 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:11.124343 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.124348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.125033 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.125035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.125820 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.125823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.126193 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.126194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.126327 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.126741 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:11.126749 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:11.128053 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:11.129502 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.129510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.129512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.129715 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.129889 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.129890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.129890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.129991 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.130248 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:11.130256 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:11.130372 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.130507 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.130636 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260809 16:37:11.131353 21848 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:11.131446 21848 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:11.131614 21848 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:11.132160 21540 ts_manager.cc:194] Registered new tserver with Master: 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:11.132923 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:33183
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:37:11.153463 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:11.234194 21858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:11.234385 21858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:11.234416 21858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:11.235549 21858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:11.235589 21858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:11.235623 21858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:11.237042 21858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:11.237260 21858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:11.237460 21858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:11.242128 21864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.242092 21872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.242103 21866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:11.242229 21858 server_base.cc:1034] running on GCE node
I20260809 16:37:11.242527 21858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:11.242743 21858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:11.243877 21858 hybrid_clock.cc:648] HybridClock initialized: now 1786293431243856 us; error 32 us; skew 500 ppm
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:11.246196 21858 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:11.246599 21858 webserver.cc:460] Webserver started at http://127.31.15.194:37195/ using document root <none> and password file <none>
I20260809 16:37:11.246784 21858 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:11.246830 21858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:11.246950 21858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:11.247974 21858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "77c2639ae9f84599bedcedb42a60f819"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "bbc4eff3b5b8b4c333023c2a432da46c"
server_key_iv: "6d9d33e6d58b61ac9e568d7b12607949"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.248312 21858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "77c2639ae9f84599bedcedb42a60f819"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "bbc4eff3b5b8b4c333023c2a432da46c"
server_key_iv: "6d9d33e6d58b61ac9e568d7b12607949"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.249490 21858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:37:11.250145 21881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.250294 21858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:11.250376 21858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "77c2639ae9f84599bedcedb42a60f819"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "bbc4eff3b5b8b4c333023c2a432da46c"
server_key_iv: "6d9d33e6d58b61ac9e568d7b12607949"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.250434 21858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:11.260854 21858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:11.262416 21858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:11.262537 21858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:11.262724 21858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:11.262984 21858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:11.263022 21858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.263049 21858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:11.263072 21858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.268477 21858 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:45425
I20260809 16:37:11.268544 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:45425 every 8 connection(s)
I20260809 16:37:11.268803 21858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:11.276904 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.269412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:42603 (local address 127.31.15.254:38255)
0809 16:37:11.269463 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.269466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.269649 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:37:11.270084 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.270085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.270094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.270145 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:11.271028 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.271591 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.272475 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.272663 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.272705 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.274658 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:11.274660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:11.274660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:11.274666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:11.275773 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.276137 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.276138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.276139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.276200 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.276392 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.276392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.276393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.276497 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:11.276544 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.276776 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.276841 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:37:11.277063 22069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.269374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:42603)
0809 16:37:11.269535 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.269574 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.270012 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.270204 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.270208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.270748 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:11.270962 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.270968 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.271664 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.271665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.272417 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.272419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.272712 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.272713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.272804 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.273235 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:11.273243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:11.274575 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:11.275837 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.275843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.275845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.276050 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.276259 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.276259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.276259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.276342 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.276547 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:11.276552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:11.276679 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.276759 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.276869 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260809 16:37:11.277523 22067 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:11.277611 22067 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:11.277818 22067 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:11.277828 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 21858
I20260809 16:37:11.277932 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:37:11.278113 31807 external_mini_cluster.cc:1428] Setting key 91eec5d99f929ee91928160069078e46
I20260809 16:37:11.278252 21540 ts_manager.cc:194] Registered new tserver with Master: 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425)
I20260809 16:37:11.278848 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:42603
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:37:11.301430 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:11.380553 22076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:11.380753 22076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:11.380775 22076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:11.381886 22076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:11.381930 22076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:11.381955 22076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:11.383375 22076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:11.383603 22076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:11.383801 22076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:11.388559 22091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.388586 22090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:11.388616 22076 server_base.cc:1034] running on GCE node
W20260809 16:37:11.388599 22094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:11.388986 22076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:11.389243 22076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:11.390390 22076 hybrid_clock.cc:648] HybridClock initialized: now 1786293431390379 us; error 30 us; skew 500 ppm
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:11.392777 22076 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:11.393196 22076 webserver.cc:460] Webserver started at http://127.31.15.195:35693/ using document root <none> and password file <none>
I20260809 16:37:11.393401 22076 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:11.393450 22076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:11.393573 22076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:11.394531 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "f9354705623e4cd2b8fbfeaa06c0994e"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "1ecd95d7975c4339f058d5eb9352753d"
server_key_iv: "98f2f08c88b873564e0cf2df309f05cb"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.394855 22076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "f9354705623e4cd2b8fbfeaa06c0994e"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "1ecd95d7975c4339f058d5eb9352753d"
server_key_iv: "98f2f08c88b873564e0cf2df309f05cb"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.396061 22076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:37:11.396746 22105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.396910 22076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:11.396992 22076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "f9354705623e4cd2b8fbfeaa06c0994e"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "1ecd95d7975c4339f058d5eb9352753d"
server_key_iv: "98f2f08c88b873564e0cf2df309f05cb"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.397047 22076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:11.411211 22076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:11.412907 22076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:11.413056 22076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:11.413262 22076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:11.413542 22076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:11.413575 22076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.413604 22076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:11.413628 22076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.419039 22076 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:36331
I20260809 16:37:11.419102 22290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:36331 every 8 connection(s)
I20260809 16:37:11.419415 22076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:11.425788 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 22076
I20260809 16:37:11.425889 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:37:11.426048 31807 external_mini_cluster.cc:1428] Setting key 34e7bffdbd766913da72ffc1b9785f17
I20260809 16:37:11.428112 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.420029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:57281 (local address 127.31.15.254:38255)
0809 16:37:11.420069 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.420072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.420264 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:37:11.420733 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.420734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.420746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.420805 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:11.421725 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.422279 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.423288 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.423479 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.423545 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.425456 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:11.425458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:11.425458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:11.425465 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:11.426817 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.427218 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.427219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.427220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.427309 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.427537 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.427537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.427538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.427644 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:11.427700 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.427945 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.428021 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:37:11.428267 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.420016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:57281)
0809 16:37:11.420149 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.420183 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.420631 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.420889 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.420894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.421429 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:11.421651 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.421656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.422343 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.422344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.423197 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.423200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.423554 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.423555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.423690 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.424093 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:11.424101 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:11.425371 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:11.426889 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.426895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.426898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.427134 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.427372 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:11.427373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:11.427373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:11.427482 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:11.427699 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:11.427705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:11.427832 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.427924 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.428070 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":72,"threads_started":1}
I20260809 16:37:11.428683 22291 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:11.428768 22291 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:11.428925 22291 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:11.429263 21540 ts_manager.cc:194] Registered new tserver with Master: f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
I20260809 16:37:11.429610 21540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:57281
I20260809 16:37:11.437757 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:11.444811 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.439046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.31.15.254:38255)
0809 16:37:11.439105 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.439107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.439206 (+ 99us) server_negotiation.cc:407] Connection header received
0809 16:37:11.439295 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.439296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.439307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.439362 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:11.439940 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.440425 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.441134 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.441320 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.441388 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.442512 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:11.442513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:11.442514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:11.442521 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:11.443776 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.444088 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.444089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.444089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.444138 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:11.444295 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:11.444296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:11.444296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:11.444436 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:11.444525 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.444648 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.444724 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:37:11.446121 21540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:37:11.447770 21540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:37:11.454519 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.0.0.1:45972)
0809 16:37:11.450647 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.450677 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.450750 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.450978 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.450981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.450983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:11.451053 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.451055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452107 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.452108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.453297 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.453300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454213 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.454306 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454320 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.454392 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.454448 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":77,"threads_started":1}
I20260809 16:37:11.454536 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.31.15.193:43903)
0809 16:37:11.450479 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.450484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.450757 (+ 273us) server_negotiation.cc:407] Connection header received
0809 16:37:11.450853 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.450856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.450873 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.451139 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:11.451213 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452010 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.453397 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454139 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454203 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454265 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.454406 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.454447 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":77,"threads_started":1}
I20260809 16:37:11.454895 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:36331 (local address 127.0.0.1:46458)
0809 16:37:11.451018 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.451027 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.451078 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.451242 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.451243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.451246 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:11.451309 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.451311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452588 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.452589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.453610 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.453612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454612 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.454682 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454706 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.454758 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.454804 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":4,"thread_start_us":78,"threads_started":1}
I20260809 16:37:11.454909 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.31.15.195:36331)
0809 16:37:11.450428 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.450432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.451078 (+ 646us) server_negotiation.cc:407] Connection header received
0809 16:37:11.451144 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.451146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.451160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.451235 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:11.451375 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452480 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.453700 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454443 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454516 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454573 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.454785 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.454832 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260809 16:37:11.455345 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54122 (local address 127.31.15.194:45425)
0809 16:37:11.450627 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.450630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.450959 (+ 329us) server_negotiation.cc:407] Connection header received
0809 16:37:11.451010 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.451012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.451028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.451103 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:11.451346 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452475 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.453681 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454441 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454516 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454573 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.455199 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.455247 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260809 16:37:11.455583 22178 tablet_service.cc:1467] Processing CreateTablet for tablet 16fa542e63d24868a9241de920f4bd35 (DEFAULT_TABLE table=test-table [id=c676b87ff9494510bf134c101d580066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:11.455579 21736 tablet_service.cc:1467] Processing CreateTablet for tablet 16fa542e63d24868a9241de920f4bd35 (DEFAULT_TABLE table=test-table [id=c676b87ff9494510bf134c101d580066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:11.455965 22178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fa542e63d24868a9241de920f4bd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:11.455966 21736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fa542e63d24868a9241de920f4bd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:11.459390 22323 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Bootstrap starting.
I20260809 16:37:11.459434 22322 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap starting.
I20260809 16:37:11.455154 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.450016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:45425 (local address 127.0.0.1:54122)
0809 16:37:11.450869 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.450888 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.450938 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.451191 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.451192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.451194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:11.451251 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.451252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.452591 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.452592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.453586 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.453588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.454614 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.454615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.454682 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.454710 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.454764 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.454805 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":128,"thread_start_us":59,"threads_started":1}
I20260809 16:37:11.460116 22323 tablet_bootstrap.cc:654] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:11.460116 22322 tablet_bootstrap.cc:654] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:11.460302 21959 tablet_service.cc:1467] Processing CreateTablet for tablet 16fa542e63d24868a9241de920f4bd35 (DEFAULT_TABLE table=test-table [id=c676b87ff9494510bf134c101d580066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:11.460618 21959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16fa542e63d24868a9241de920f4bd35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:11.460714 22323 log.cc:824] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:11.460737 22322 log.cc:824] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:11.461521 22323 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: No bootstrap required, opened a new log
I20260809 16:37:11.461591 22323 ts_tablet_manager.cc:1397] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:11.462817 22322 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: No bootstrap required, opened a new log
I20260809 16:37:11.462903 22322 ts_tablet_manager.cc:1397] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 16:37:11.463415 22323 raft_consensus.cc:348] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.463584 22323 raft_consensus.cc:374] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:11.463620 22323 raft_consensus.cc:729] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9354705623e4cd2b8fbfeaa06c0994e, State: Initialized, Role: FOLLOWER
I20260809 16:37:11.463740 22323 consensus_queue.cc:260] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.464036 22323 ts_tablet_manager.cc:1428] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:37:11.464071 22291 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
I20260809 16:37:11.464673 22322 raft_consensus.cc:348] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.464798 22322 raft_consensus.cc:374] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:11.464828 22322 raft_consensus.cc:729] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849f607764fa4aeea15e11dafbf38ef1, State: Initialized, Role: FOLLOWER
I20260809 16:37:11.464943 22322 consensus_queue.cc:260] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.465237 21848 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
I20260809 16:37:11.465421 22322 ts_tablet_manager.cc:1428] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:37:11.465597 22327 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Bootstrap starting.
I20260809 16:37:11.466357 22327 tablet_bootstrap.cc:654] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:11.466786 22327 log.cc:824] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:11.467489 22327 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: No bootstrap required, opened a new log
I20260809 16:37:11.467549 22327 ts_tablet_manager.cc:1397] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:37:11.468845 22327 raft_consensus.cc:348] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.468983 22327 raft_consensus.cc:374] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:11.469017 22327 raft_consensus.cc:729] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c2639ae9f84599bedcedb42a60f819, State: Initialized, Role: FOLLOWER
I20260809 16:37:11.469138 22327 consensus_queue.cc:260] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.469367 22327 ts_tablet_manager.cc:1428] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:11.469452 22067 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
W20260809 16:37:11.519359 22068 tablet.cc:2369] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:11.622651 21849 tablet.cc:2369] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:11.669946 22293 tablet.cc:2369] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:11.706292 22329 raft_consensus.cc:482] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:11.706423 22329 raft_consensus.cc:504] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.706808 22329 leader_election.cc:288] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:11.709487 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.707001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.31.15.195:43595)
0809 16:37:11.707061 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.707074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.707126 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.707296 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.707296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.707298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:11.707362 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.707364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.707991 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.707992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.708745 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.708747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.709256 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.709257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.709324 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.709327 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.709371 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.709409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":20}
I20260809 16:37:11.709487 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.707117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:43595 (local address 127.31.15.193:43903)
0809 16:37:11.707172 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.707174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.707182 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:37:11.707211 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.707211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.707219 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.707283 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:11.707444 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.707914 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.708830 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.709186 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.709248 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.709258 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.709386 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.709428 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:37:11.709597 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.707076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:48447 (local address 127.31.15.194:45425)
0809 16:37:11.707121 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.707123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.707289 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:37:11.707351 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.707352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.707362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.707409 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:11.707554 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.708010 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.708912 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.709292 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.709352 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.709360 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.709496 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.709531 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:37:11.709599 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.707003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:45425 (local address 127.31.15.195:48447)
0809 16:37:11.707209 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.707219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:11.707296 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:11.707422 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:11.707422 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:11.707424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:11.707492 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.707493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.708092 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.708092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.708823 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:11.708824 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.709369 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.709370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:11.709433 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.709437 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:37:11.709481 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:37:11.709517 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":55,"thread_start_us":96,"threads_started":1}
I20260809 16:37:11.709838 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fa542e63d24868a9241de920f4bd35" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849f607764fa4aeea15e11dafbf38ef1" is_pre_election: true
I20260809 16:37:11.709921 21990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fa542e63d24868a9241de920f4bd35" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c2639ae9f84599bedcedb42a60f819" is_pre_election: true
I20260809 16:37:11.709998 21767 raft_consensus.cc:2454] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 0.
I20260809 16:37:11.710067 21990 raft_consensus.cc:2454] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 0.
I20260809 16:37:11.710233 22107 leader_election.cc:302] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1, f9354705623e4cd2b8fbfeaa06c0994e; no voters:
I20260809 16:37:11.710382 22329 raft_consensus.cc:2781] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:11.710446 22329 raft_consensus.cc:482] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:11.710476 22329 raft_consensus.cc:3037] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:11.711156 22329 raft_consensus.cc:504] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.711297 22329 leader_election.cc:288] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 election: Requested vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:11.711447 21990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fa542e63d24868a9241de920f4bd35" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c2639ae9f84599bedcedb42a60f819"
I20260809 16:37:11.711459 21767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16fa542e63d24868a9241de920f4bd35" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849f607764fa4aeea15e11dafbf38ef1"
I20260809 16:37:11.711522 21767 raft_consensus.cc:3037] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:11.711524 21990 raft_consensus.cc:3037] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:11.712379 21767 raft_consensus.cc:2454] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 1.
I20260809 16:37:11.712419 21990 raft_consensus.cc:2454] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 1.
I20260809 16:37:11.712568 22109 leader_election.cc:302] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e; no voters:
I20260809 16:37:11.712666 22329 raft_consensus.cc:2781] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:11.712812 22329 raft_consensus.cc:686] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 LEADER]: Becoming Leader. State: Replica: f9354705623e4cd2b8fbfeaa06c0994e, State: Running, Role: LEADER
I20260809 16:37:11.712908 22329 consensus_queue.cc:237] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:11.713703 21540 catalog_manager.cc:5495] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e reported cstate change: term changed from 0 to 1, leader changed from <none> to f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195). New cstate: current_term: 1 leader_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } health_report { overall_health: HEALTHY } } }
I20260809 16:37:11.734117 22315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.731959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46466 (local address 127.31.15.195:36331)
0809 16:37:11.732003 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:11.732005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:11.732013 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:37:11.732072 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:11.732073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:11.732100 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:11.732155 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:11.732272 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.732713 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.733432 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:11.733616 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:11.733670 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:11.733809 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:11.733902 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:11.733969 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:11.734004 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:37:11.734025 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:37:11.735769 21767 raft_consensus.cc:1261] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Refusing update from remote peer f9354705623e4cd2b8fbfeaa06c0994e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:11.735770 21990 raft_consensus.cc:1261] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Refusing update from remote peer f9354705623e4cd2b8fbfeaa06c0994e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:11.736095 22337 consensus_queue.cc:1035] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Connected to new peer: Peer: permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:11.736213 22329 consensus_queue.cc:1035] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:11.738884 22344 mvcc.cc:205] Tried to move back new op lower bound from 7316657896387268608 to 7316657896296517632. Current Snapshot: MvccSnapshot[applied={T|T < 7316657896387268608 or (T in {7316657896387268608})}]
I20260809 16:37:11.740778 21539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:11.740860 21539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:11.741564 21539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Sending DeleteTablet for 3 replicas of tablet 16fa542e63d24868a9241de920f4bd35
I20260809 16:37:11.741887 22178 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fa542e63d24868a9241de920f4bd35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:11 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:46458
I20260809 16:37:11.741921 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 21636
I20260809 16:37:11.741914 21736 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fa542e63d24868a9241de920f4bd35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:11 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45972
I20260809 16:37:11.744668 22362 tablet_replica.cc:330] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: stopping tablet replica
I20260809 16:37:11.744648 21959 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fa542e63d24868a9241de920f4bd35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:11 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:54122
I20260809 16:37:11.744843 22362 raft_consensus.cc:2229] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:37:11.744989 22362 raft_consensus.cc:2258] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:11.748019 22364 tablet_replica.cc:330] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: stopping tablet replica
I20260809 16:37:11.748131 22364 raft_consensus.cc:2229] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:11.748234 22364 raft_consensus.cc:2258] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:11.749341 22364 ts_tablet_manager.cc:1905] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:11.749430 22362 ts_tablet_manager.cc:1905] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:11.750857 22364 ts_tablet_manager.cc:1918] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:11.750931 22364 log.cc:1196] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/16fa542e63d24868a9241de920f4bd35
I20260809 16:37:11.751219 22364 ts_tablet_manager.cc:1939] T 16fa542e63d24868a9241de920f4bd35 P 77c2639ae9f84599bedcedb42a60f819: Deleting consensus metadata
I20260809 16:37:11.751415 22362 ts_tablet_manager.cc:1918] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:11.751541 22362 log.cc:1196] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/16fa542e63d24868a9241de920f4bd35
I20260809 16:37:11.751801 21519 catalog_manager.cc:4841] TS 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): tablet 16fa542e63d24868a9241de920f4bd35 (table test-table [id=c676b87ff9494510bf134c101d580066]) successfully deleted
I20260809 16:37:11.751959 22362 ts_tablet_manager.cc:1939] T 16fa542e63d24868a9241de920f4bd35 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting consensus metadata
I20260809 16:37:11.752444 21515 catalog_manager.cc:4841] TS f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): tablet 16fa542e63d24868a9241de920f4bd35 (table test-table [id=c676b87ff9494510bf134c101d580066]) successfully deleted
W20260809 16:37:11.762728 21513 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:43903 (error 108)
I20260809 16:37:11.762964 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 21858
I20260809 16:37:11.763414 22311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:11.763248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.0.0.1:45986)
0809 16:37:11.763328 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:11.763381 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 16:37:11.767130 21513 catalog_manager.cc:4584] TS 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16fa542e63d24868a9241de920f4bd35: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43903: connect: Connection refused (error 111)
I20260809 16:37:11.770890 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 22076
I20260809 16:37:11.783797 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 21484
I20260809 16:37:11.799371 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--webserver_interface=127.31.15.254
--webserver_port=41641
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:11.880201 22371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:11.880406 22371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:11.880436 22371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:11.881565 22371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:11.881605 22371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:11.881618 22371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:11.881642 22371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:11.881665 22371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:11.883138 22371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=41641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:11.883389 22371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:11.883613 22371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:11.894306 22390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.894308 22382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:11.894306 22385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:11.894627 22371 server_base.cc:1034] running on GCE node
I20260809 16:37:11.894887 22371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:11.895186 22371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:11.896322 22371 hybrid_clock.cc:648] HybridClock initialized: now 1786293431896318 us; error 38 us; skew 500 ppm
Aug 09 16:37:11 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:11.898794 22371 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:11.899207 22371 webserver.cc:460] Webserver started at http://127.31.15.254:41641/ using document root <none> and password file <none>
I20260809 16:37:11.899433 22371 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:11.899499 22371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:11.900835 22371 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:11.901482 22408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:11.901650 22371 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:11.901736 22371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4"
format_stamp: "Formatted at 2026-08-09 16:37:10 on dist-test-slave-21b9"
server_key: "5c9887e03db8933bd26339ec41c81280"
server_key_iv: "fe714281f6eb5b7e1bb540f107fd7882"
server_key_version: "encryptionkey@0"
I20260809 16:37:11.902109 22371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:11.917673 22371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:11.919528 22371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:11.919672 22371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:11.923494 22371 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38255
I20260809 16:37:11.923534 22485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38255 every 8 connection(s)
I20260809 16:37:11.923907 22371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:11.924311 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 22371
I20260809 16:37:11.924561 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:43903
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=38761
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:11.925670 22487 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:37:11.926564 22487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap starting.
I20260809 16:37:11.935847 22487 log.cc:824] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:11.937960 22487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:11.938251 22487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap complete.
I20260809 16:37:11.940572 22487 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:11.940805 22487 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Initialized, Role: FOLLOWER
I20260809 16:37:11.940968 22487 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:11.941037 22487 raft_consensus.cc:388] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:11.941087 22487 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:11.941140 22487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:37:11.942175 22487 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:11.942271 22487 leader_election.cc:302] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ded5a6fa9a8d4cbd9bc413bebd4d9cf4; no voters:
I20260809 16:37:11.942437 22487 leader_election.cc:288] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:37:11.942493 22494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:37:11.942620 22494 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 LEADER]: Becoming Leader. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Running, Role: LEADER
I20260809 16:37:11.942713 22487 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:11.942720 22494 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:11.943097 22501 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:11.943109 22495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded5a6fa9a8d4cbd9bc413bebd4d9cf4. Latest consensus state: current_term: 2 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:11.943264 22501 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:11.943308 22495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:11.944582 22520 catalog_manager.cc:1259] Loaded cluster ID: a4b8f126917a410395324c08349b2590
I20260809 16:37:11.944638 22520 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: loading cluster ID for follower catalog manager: success
W20260809 16:37:11.944835 22520 catalog_manager.cc:1576] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260809 16:37:11.944881 22520 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260809 16:37:11.944926 22511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:11.945272 22511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:11.945358 22511 catalog_manager.cc:1259] Loaded cluster ID: a4b8f126917a410395324c08349b2590
I20260809 16:37:11.945403 22511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:37:11.945562 22511 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 16:37:11.949807 22511 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 16:37:11.952200 22511 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:11.952401 22511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Loaded TSK: 0
I20260809 16:37:11.952621 22511 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:37:12.008473 22489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:12.008749 22489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:12.008785 22489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:12.010491 22489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:12.010545 22489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:12.010583 22489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:12.012616 22489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:43903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=38761
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:12.012848 22489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:12.013104 22489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:12.019043 22535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.019121 22537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.019043 22532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.019869 22489 server_base.cc:1034] running on GCE node
I20260809 16:37:12.020063 22489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:12.020304 22489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:12.021457 22489 hybrid_clock.cc:648] HybridClock initialized: now 1786293432021446 us; error 31 us; skew 500 ppm
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:12.023857 22489 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:12.024302 22489 webserver.cc:460] Webserver started at http://127.31.15.193:38761/ using document root <none> and password file <none>
I20260809 16:37:12.024520 22489 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:12.024572 22489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:12.025859 22489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:12.026477 22549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.026644 22489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:12.026727 22489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "849f607764fa4aeea15e11dafbf38ef1"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "084bb4ab40a70af56fb822b571cbb159"
server_key_iv: "84bde4f3e343fbfabf502a9469308db7"
server_key_version: "encryptionkey@0"
I20260809 16:37:12.027045 22489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:12.049980 22489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:12.052199 22489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:12.052315 22489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:12.052557 22489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:12.053049 22563 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:37:12.054458 22489 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:37:12.054510 22489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:37:12.054534 22489 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:37:12.055163 22489 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:37:12.055207 22489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:12.055291 22563 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap starting.
I20260809 16:37:12.063732 22489 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:43903
I20260809 16:37:12.064258 22489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:12.066929 22563 log.cc:824] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:12.067196 22742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:43903 every 8 connection(s)
I20260809 16:37:12.069689 22563 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:12.069967 22563 tablet_bootstrap.cc:492] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap complete.
I20260809 16:37:12.070606 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 22489
I20260809 16:37:12.070915 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:45425
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=37195
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:12.071209 22563 ts_tablet_manager.cc:1397] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260809 16:37:12.072526 22563 raft_consensus.cc:348] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:12.072820 22563 raft_consensus.cc:729] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849f607764fa4aeea15e11dafbf38ef1, State: Initialized, Role: FOLLOWER
I20260809 16:37:12.072957 22563 consensus_queue.cc:260] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } }
I20260809 16:37:12.073231 22563 ts_tablet_manager.cc:1428] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:12.079463 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.067768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:57773)
0809 16:37:12.067953 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.068004 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.068710 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.069212 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.069217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.069603 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:12.069869 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.069875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.071487 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.071488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.072720 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.072722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.073511 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.073512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.073661 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.074186 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:12.074196 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:12.075924 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:12.077619 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.077626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.077629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.077925 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.078222 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.078223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.078223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.078346 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.078764 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:12.078773 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:12.078916 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.079029 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.079205 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260809 16:37:12.079499 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.068033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:57773 (local address 127.31.15.254:38255)
0809 16:37:12.068386 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.068395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.068435 (+ 40us) server_negotiation.cc:407] Connection header received
0809 16:37:12.068947 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.068954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.069015 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.069177 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:12.069999 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.071387 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.072815 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.073427 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.073506 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.076034 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:12.076037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:12.076038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:12.076048 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:12.077536 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.078024 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.078028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.078028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.078154 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.078419 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.078419 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.078420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.078699 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:12.078777 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.079117 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.079262 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":97,"threads_started":1}
I20260809 16:37:12.080426 22743 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:12.080518 22743 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:12.080693 22743 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:12.081473 22437 ts_manager.cc:194] Registered new tserver with Master: 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:12.082659 22437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:57773
I20260809 16:37:12.083532 22743 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293431, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:37:12.091114 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.082140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.0.0.1:45996)
0809 16:37:12.082578 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.082615 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.082681 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.083171 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.083173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.083533 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:12.083609 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.083611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.084816 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.084817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.085708 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.085711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.086165 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.086166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.086253 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.086717 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:12.086722 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:12.088222 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:12.089925 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.089933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.089935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.090138 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.090345 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.090346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.090347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.090414 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.090713 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:12.090721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:12.090777 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.090905 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.091037 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":95,"threads_started":1}
I20260809 16:37:12.091185 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.082412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.31.15.193:43903)
0809 16:37:12.082863 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.082868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.082879 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:12.083010 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.083012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.083030 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.083076 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:12.083707 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.084718 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.085798 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.086087 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.086153 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.088314 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:12.088323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:12.088323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:12.088331 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:12.089836 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.090221 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.090224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.090224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.090283 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.090470 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.090471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.090472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.090620 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:12.090702 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.090969 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.091089 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":77,"threads_started":1}
I20260809 16:37:12.091621 22625 tablet_service.cc:1514] Processing DeleteTablet for tablet 16fa542e63d24868a9241de920f4bd35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:11 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45996
I20260809 16:37:12.092209 22754 tablet_replica.cc:330] stopping tablet replica
I20260809 16:37:12.092321 22754 raft_consensus.cc:2229] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:12.092425 22754 raft_consensus.cc:2258] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:12.092736 22754 ts_tablet_manager.cc:1905] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:12.094635 22754 ts_tablet_manager.cc:1918] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:12.094712 22754 log.cc:1196] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/16fa542e63d24868a9241de920f4bd35
I20260809 16:37:12.094926 22754 ts_tablet_manager.cc:1939] T 16fa542e63d24868a9241de920f4bd35 P 849f607764fa4aeea15e11dafbf38ef1: Deleting consensus metadata
I20260809 16:37:12.095441 22412 catalog_manager.cc:4841] TS 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903): tablet 16fa542e63d24868a9241de920f4bd35 (table test-table [id=c676b87ff9494510bf134c101d580066]) successfully deleted
W20260809 16:37:12.160454 22749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:12.160656 22749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:12.160678 22749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:12.161792 22749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:12.161832 22749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:12.161857 22749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:12.163302 22749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:45425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=37195
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:12.163527 22749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:12.163739 22749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:12.168419 22762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.168431 22765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.168426 22769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.168727 22749 server_base.cc:1034] running on GCE node
I20260809 16:37:12.168934 22749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:12.169143 22749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:12.170275 22749 hybrid_clock.cc:648] HybridClock initialized: now 1786293432170262 us; error 31 us; skew 500 ppm
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:12.172649 22749 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:12.173043 22749 webserver.cc:460] Webserver started at http://127.31.15.194:37195/ using document root <none> and password file <none>
I20260809 16:37:12.173251 22749 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:12.173313 22749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:12.174568 22749 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:12.175065 22781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.175211 22749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:12.175313 22749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "77c2639ae9f84599bedcedb42a60f819"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "bbc4eff3b5b8b4c333023c2a432da46c"
server_key_iv: "6d9d33e6d58b61ac9e568d7b12607949"
server_key_version: "encryptionkey@0"
I20260809 16:37:12.175628 22749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:12.183625 22749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:12.185274 22749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:12.185398 22749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:12.185591 22749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:12.185863 22749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:12.185897 22749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.185922 22749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:12.185948 22749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.191354 22749 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:45425
I20260809 16:37:12.191411 22966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:45425 every 8 connection(s)
I20260809 16:37:12.191720 22749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:12.200119 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.192344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:55987 (local address 127.31.15.254:38255)
0809 16:37:12.192394 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.192396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.192598 (+ 202us) server_negotiation.cc:407] Connection header received
0809 16:37:12.193059 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.193060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.193071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.193130 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:12.194010 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.194501 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.195444 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.195640 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.195688 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.197554 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:12.197555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:12.197556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:12.197562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:12.198759 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.199212 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.199214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.199214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.199294 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.199519 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.199520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.199520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.199631 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:12.199686 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.199931 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.200027 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:37:12.200260 22973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.192321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:55987)
0809 16:37:12.192475 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.192517 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.192985 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.193191 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.193194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.193753 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:12.193933 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.193938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.194558 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.194558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.195377 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.195380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.195695 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.195696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.195788 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.196205 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:12.196212 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:12.197462 (+ 1250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:12.198847 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.198854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.198857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.199146 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.199342 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.199342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.199343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.199448 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.199686 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:12.199694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:12.199819 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.199906 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.200060 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 16:37:12.200671 22968 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:12.200757 22968 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:12.200901 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 22749
I20260809 16:37:12.200923 22968 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:12.201187 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:36331
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=35693
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:12.201289 22437 ts_manager.cc:194] Registered new tserver with Master: 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425)
I20260809 16:37:12.201695 22437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:55987
W20260809 16:37:12.282284 22975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:12.282492 22975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:12.282526 22975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:12.283667 22975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:12.283708 22975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:12.283746 22975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:12.285179 22975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:36331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=35693
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:12.285410 22975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:12.285624 22975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:12.290238 22986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.290299 22975 server_base.cc:1034] running on GCE node
W20260809 16:37:12.290258 22987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.290247 22990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.290613 22975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:12.290810 22975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:12.291947 22975 hybrid_clock.cc:648] HybridClock initialized: now 1786293432291931 us; error 28 us; skew 500 ppm
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:12.294288 22975 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:12.294659 22975 webserver.cc:460] Webserver started at http://127.31.15.195:35693/ using document root <none> and password file <none>
I20260809 16:37:12.294829 22975 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:12.294879 22975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:12.296193 22975 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:12.296698 22998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.296849 22975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.296927 22975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "f9354705623e4cd2b8fbfeaa06c0994e"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "1ecd95d7975c4339f058d5eb9352753d"
server_key_iv: "98f2f08c88b873564e0cf2df309f05cb"
server_key_version: "encryptionkey@0"
I20260809 16:37:12.297230 22975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:12.310457 22975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:12.312148 22975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:12.312253 22975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:12.312424 22975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:12.312680 22975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:12.312709 22975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.312731 22975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:12.312747 22975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.317951 22975 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:36331
I20260809 16:37:12.318014 23180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:36331 every 8 connection(s)
I20260809 16:37:12.318317 22975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:12.326424 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 22975
I20260809 16:37:12.326529 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.319043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:37451 (local address 127.31.15.254:38255)
0809 16:37:12.319104 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.319106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.319296 (+ 190us) server_negotiation.cc:407] Connection header received
0809 16:37:12.319805 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.319807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.319817 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.319864 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:12.320724 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.321199 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.322070 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.322248 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.322288 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.324199 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:12.324201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:12.324202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:12.324221 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:12.325327 (+ 1106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.325686 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.325687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.325687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.325740 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.325941 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.325942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.325943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.326048 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:12.326105 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.326357 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.326440 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:37:12.326695 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.319008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:37451)
0809 16:37:12.319176 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.319211 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.319723 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.319919 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.319922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.320459 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:12.320659 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.320664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.321255 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.321255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.322012 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.322014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.322298 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.322299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.322392 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.322796 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:12.322804 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:12.324105 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:12.325400 (+ 1295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.325407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.325409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.325621 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.325793 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:12.325793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:12.325794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:12.325887 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:12.326113 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:12.326119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:12.326242 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.326327 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.326473 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260809 16:37:12.327163 23182 heartbeater.cc:347] Connected to a master server at 127.31.15.254:38255
I20260809 16:37:12.327263 23182 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:12.327457 23182 heartbeater.cc:511] Master 127.31.15.254:38255 requested a full tablet report, sending...
I20260809 16:37:12.327867 22437 ts_manager.cc:194] Registered new tserver with Master: f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
I20260809 16:37:12.328219 22437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:37451
I20260809 16:37:12.332078 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.326877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.31.15.254:38255)
0809 16:37:12.326927 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.326929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.327030 (+ 101us) server_negotiation.cc:407] Connection header received
0809 16:37:12.327085 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.327086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.327095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.327138 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:12.327751 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.328283 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.329026 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.329210 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.329261 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.330251 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:12.330252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:12.330252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:12.330258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:12.331297 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.331572 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.331573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.331573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.331617 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:12.331734 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:12.331734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:12.331735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:12.331814 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:12.331920 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.331944 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.331997 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:37:12.332746 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:12.335446 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.333044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.31.15.254:38255)
0809 16:37:12.333086 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.333087 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.333157 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:37:12.333208 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.333208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.333233 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.333285 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:12.333357 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.333773 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.334668 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.334953 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.335003 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.335161 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:12.335256 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:12.335355 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.335371 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.335388 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:37:12.335700 22437 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:37:12.337275 22437 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:37:12.339260 22625 tablet_service.cc:1467] Processing CreateTablet for tablet 68628d35a2b5449a89cac892c33689f4 (DEFAULT_TABLE table=test-table [id=e0002bfc24894426aa49e475f358f0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:12.339483 22625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68628d35a2b5449a89cac892c33689f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:12.340873 22563 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap starting.
I20260809 16:37:12.341180 22563 tablet_bootstrap.cc:654] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:12.342077 22563 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: No bootstrap required, opened a new log
I20260809 16:37:12.342128 22563 ts_tablet_manager.cc:1397] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:12.342271 22563 raft_consensus.cc:348] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.342338 22563 raft_consensus.cc:374] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:12.342383 22563 raft_consensus.cc:729] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849f607764fa4aeea15e11dafbf38ef1, State: Initialized, Role: FOLLOWER
I20260809 16:37:12.342386 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.339000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:45425 (local address 127.0.0.1:54138)
0809 16:37:12.339053 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.339061 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.339095 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.339452 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.339453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.339455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:12.339494 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.339496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.340465 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.340466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.341413 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.341415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.342141 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.342142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.342212 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.342232 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.342281 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.342323 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:37:12.342417 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.339108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54138 (local address 127.31.15.194:45425)
0809 16:37:12.339298 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.339302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.339312 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:37:12.339350 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.339352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.339367 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.339439 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:12.339555 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.340385 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.341477 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.342070 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.342132 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.342187 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.342305 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.342342 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260809 16:37:12.342425 22563 consensus_queue.cc:260] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.342516 22563 ts_tablet_manager.cc:1428] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.343228 22851 tablet_service.cc:1467] Processing CreateTablet for tablet 68628d35a2b5449a89cac892c33689f4 (DEFAULT_TABLE table=test-table [id=e0002bfc24894426aa49e475f358f0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:12.343621 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68628d35a2b5449a89cac892c33689f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:12.344915 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.339015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:36331 (local address 127.0.0.1:46472)
0809 16:37:12.339306 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.339316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.339450 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.341263 (+ 1813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.341264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.341266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:12.341302 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.341303 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.342930 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.342931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.343892 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.343895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.344759 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.344760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.344809 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.344812 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.344843 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.344870 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":52,"threads_started":1}
I20260809 16:37:12.344969 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.340903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46472 (local address 127.31.15.195:36331)
0809 16:37:12.341071 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.341075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.341086 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:12.341118 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.341120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.341135 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.341207 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:12.341360 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.342826 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.344004 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.344680 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.344752 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.344809 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.344859 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.344895 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260809 16:37:12.345608 23069 tablet_service.cc:1467] Processing CreateTablet for tablet 68628d35a2b5449a89cac892c33689f4 (DEFAULT_TABLE table=test-table [id=e0002bfc24894426aa49e475f358f0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:12.345916 23069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68628d35a2b5449a89cac892c33689f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:12.346915 23197 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Bootstrap starting.
I20260809 16:37:12.347529 23197 tablet_bootstrap.cc:654] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:12.347985 23197 log.cc:824] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:12.348670 23197 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: No bootstrap required, opened a new log
I20260809 16:37:12.348730 23197 ts_tablet_manager.cc:1397] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:37:12.348951 23200 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Bootstrap starting.
I20260809 16:37:12.349599 23200 tablet_bootstrap.cc:654] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:12.350118 23200 log.cc:824] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:12.350551 23197 raft_consensus.cc:348] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.350724 23197 raft_consensus.cc:374] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:12.350767 23197 raft_consensus.cc:729] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c2639ae9f84599bedcedb42a60f819, State: Initialized, Role: FOLLOWER
I20260809 16:37:12.350895 23200 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: No bootstrap required, opened a new log
I20260809 16:37:12.350890 23197 consensus_queue.cc:260] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.350961 23200 ts_tablet_manager.cc:1397] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:37:12.351166 23197 ts_tablet_manager.cc:1428] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:37:12.351193 22968 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
I20260809 16:37:12.352468 23200 raft_consensus.cc:348] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.352588 23200 raft_consensus.cc:374] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:12.352610 23200 raft_consensus.cc:729] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9354705623e4cd2b8fbfeaa06c0994e, State: Initialized, Role: FOLLOWER
I20260809 16:37:12.352718 23200 consensus_queue.cc:260] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.352941 23200 ts_tablet_manager.cc:1428] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:12.352970 23182 heartbeater.cc:503] Master 127.31.15.254:38255 was elected leader, sending a full tablet report...
W20260809 16:37:12.442268 22970 tablet.cc:2369] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:12.568663 22745 tablet.cc:2369] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:12.568912 23183 tablet.cc:2369] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:12.642643 23205 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:12.642771 23205 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.643217 23205 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:12.646170 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.643475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:37947 (local address 127.31.15.194:45425)
0809 16:37:12.643524 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.643526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.643747 (+ 221us) server_negotiation.cc:407] Connection header received
0809 16:37:12.643782 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.643784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.643794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.643845 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:12.644008 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.644522 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645467 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.645847 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645902 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.645912 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.646063 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.646099 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:37:12.646183 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.643418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.31.15.195:55439)
0809 16:37:12.643486 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.643499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.643547 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.643840 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.643841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.643843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:12.643919 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.643921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.644632 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.644633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.645371 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.645373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.645941 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.646014 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.646016 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.646071 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.646106 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 16:37:12.646198 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.643529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:55439 (local address 127.31.15.193:43903)
0809 16:37:12.643690 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.643691 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.643701 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:37:12.643731 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.643732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.643756 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.643827 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:12.643986 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.644553 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645458 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.645862 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645934 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.646019 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.646074 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.646115 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":62,"threads_started":1}
I20260809 16:37:12.646193 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.643418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:45425 (local address 127.31.15.195:37947)
0809 16:37:12.643681 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.643689 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:12.643734 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:12.643871 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:12.643872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:12.643874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:12.643939 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.643941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.644603 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.644604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.645382 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:12.645385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.645925 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.645926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:12.645994 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.645997 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:12.646051 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:37:12.646091 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":99,"thread_start_us":112,"threads_started":1}
I20260809 16:37:12.646495 22645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68628d35a2b5449a89cac892c33689f4" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849f607764fa4aeea15e11dafbf38ef1" is_pre_election: true
I20260809 16:37:12.646657 22645 raft_consensus.cc:2454] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 0.
I20260809 16:37:12.646761 22892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68628d35a2b5449a89cac892c33689f4" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c2639ae9f84599bedcedb42a60f819" is_pre_election: true
I20260809 16:37:12.646865 23002 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1, f9354705623e4cd2b8fbfeaa06c0994e; no voters:
I20260809 16:37:12.646909 22892 raft_consensus.cc:2454] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 0.
I20260809 16:37:12.646982 23205 raft_consensus.cc:2781] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:12.647037 23205 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:12.647060 23205 raft_consensus.cc:3037] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:12.647770 23205 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.647887 23205 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 election: Requested vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903)
I20260809 16:37:12.648021 22645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68628d35a2b5449a89cac892c33689f4" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849f607764fa4aeea15e11dafbf38ef1"
I20260809 16:37:12.648082 22892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68628d35a2b5449a89cac892c33689f4" candidate_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77c2639ae9f84599bedcedb42a60f819"
I20260809 16:37:12.648103 22645 raft_consensus.cc:3037] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:12.648168 22892 raft_consensus.cc:3037] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:12.648741 22645 raft_consensus.cc:2454] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 1.
I20260809 16:37:12.648907 23002 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1, f9354705623e4cd2b8fbfeaa06c0994e; no voters:
I20260809 16:37:12.648977 22892 raft_consensus.cc:2454] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9354705623e4cd2b8fbfeaa06c0994e in term 1.
I20260809 16:37:12.648988 23205 raft_consensus.cc:2781] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:12.649164 23205 raft_consensus.cc:686] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 LEADER]: Becoming Leader. State: Replica: f9354705623e4cd2b8fbfeaa06c0994e, State: Running, Role: LEADER
I20260809 16:37:12.649259 23205 consensus_queue.cc:237] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:12.650074 22435 catalog_manager.cc:5495] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e reported cstate change: term changed from 0 to 1, leader changed from <none> to f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195). New cstate: current_term: 1 leader_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } health_report { overall_health: UNKNOWN } } }
I20260809 16:37:12.694523 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.692484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46482 (local address 127.31.15.195:36331)
0809 16:37:12.692541 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:12.692542 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:12.692550 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:37:12.692625 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:12.692627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:12.692649 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:12.692725 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:12.692806 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.693263 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.693919 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:12.694101 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:12.694145 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:12.694245 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:12.694340 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:12.694395 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:12.694408 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 16:37:12.694423 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:37:12.696358 22645 raft_consensus.cc:1261] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Refusing update from remote peer f9354705623e4cd2b8fbfeaa06c0994e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:12.696359 22892 raft_consensus.cc:1261] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Refusing update from remote peer f9354705623e4cd2b8fbfeaa06c0994e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:12.696668 23205 consensus_queue.cc:1035] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Connected to new peer: Peer: permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:12.696810 23210 consensus_queue.cc:1035] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [LEADER]: Connected to new peer: Peer: permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:12.699075 23213 mvcc.cc:205] Tried to move back new op lower bound from 7316657900321685504 to 7316657900131926016. Current Snapshot: MvccSnapshot[applied={T|T < 7316657900321685504}]
I20260809 16:37:12.701081 22435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:12.701161 22435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:12.701820 22435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Sending DeleteTablet for 3 replicas of tablet 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:12.702100 23069 tablet_service.cc:1514] Processing DeleteTablet for tablet 68628d35a2b5449a89cac892c33689f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:12 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:46472
I20260809 16:37:12.702154 22625 tablet_service.cc:1514] Processing DeleteTablet for tablet 68628d35a2b5449a89cac892c33689f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:12 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:45996
I20260809 16:37:12.702145 22851 tablet_service.cc:1514] Processing DeleteTablet for tablet 68628d35a2b5449a89cac892c33689f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:12 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:54138
I20260809 16:37:12.702350 23223 tablet_replica.cc:330] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: stopping tablet replica
I20260809 16:37:12.702342 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 22489
I20260809 16:37:12.702400 23227 tablet_replica.cc:330] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: stopping tablet replica
I20260809 16:37:12.702495 23227 raft_consensus.cc:2229] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:12.704911 23227 raft_consensus.cc:2258] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:12.708710 23223 raft_consensus.cc:2229] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:37:12.708873 23223 raft_consensus.cc:2258] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:12.709370 23223 ts_tablet_manager.cc:1905] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:12.710889 23223 ts_tablet_manager.cc:1918] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:12.711009 23223 log.cc:1196] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/68628d35a2b5449a89cac892c33689f4
I20260809 16:37:12.711344 23223 ts_tablet_manager.cc:1939] T 68628d35a2b5449a89cac892c33689f4 P f9354705623e4cd2b8fbfeaa06c0994e: Deleting consensus metadata
I20260809 16:37:12.711740 22413 catalog_manager.cc:4841] TS f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): tablet 68628d35a2b5449a89cac892c33689f4 (table test-table [id=e0002bfc24894426aa49e475f358f0c2]) successfully deleted
I20260809 16:37:12.713644 23227 ts_tablet_manager.cc:1905] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:12.714892 23227 ts_tablet_manager.cc:1918] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:12.714968 23227 log.cc:1196] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/68628d35a2b5449a89cac892c33689f4
I20260809 16:37:12.715204 23227 ts_tablet_manager.cc:1939] T 68628d35a2b5449a89cac892c33689f4 P 77c2639ae9f84599bedcedb42a60f819: Deleting consensus metadata
I20260809 16:37:12.715626 22415 catalog_manager.cc:4841] TS 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): tablet 68628d35a2b5449a89cac892c33689f4 (table test-table [id=e0002bfc24894426aa49e475f358f0c2]) successfully deleted
W20260809 16:37:12.718652 22412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:43903 (error 108)
I20260809 16:37:12.719048 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 22749
I20260809 16:37:12.719136 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:12.718994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:43903 (local address 127.0.0.1:46000)
0809 16:37:12.719042 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:12.719099 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:37:12.724180 22412 catalog_manager.cc:4584] TS 849f607764fa4aeea15e11dafbf38ef1 (127.31.15.193:43903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68628d35a2b5449a89cac892c33689f4: Network error: Client connection negotiation failed: client connection to 127.31.15.193:43903: connect: Connection refused (error 111)
I20260809 16:37:12.732606 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 22975
I20260809 16:37:12.743234 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 22371
I20260809 16:37:12.755554 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--webserver_interface=127.31.15.254
--webserver_port=41641
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:12.835739 23235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:12.835929 23235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:12.835950 23235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:12.837080 23235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:12.837124 23235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:12.837137 23235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:12.837147 23235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:12.837158 23235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:12.838732 23235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=41641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:12.838965 23235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:12.839182 23235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:12.843804 23249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.843802 23243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.843876 23235 server_base.cc:1034] running on GCE node
W20260809 16:37:12.843801 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.844283 23235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:12.844575 23235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:12.845731 23235 hybrid_clock.cc:648] HybridClock initialized: now 1786293432845721 us; error 28 us; skew 500 ppm
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:12.848131 23235 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:12.848537 23235 webserver.cc:460] Webserver started at http://127.31.15.254:41641/ using document root <none> and password file <none>
I20260809 16:37:12.848755 23235 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:12.848799 23235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:12.850116 23235 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:37:12.850744 23260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.850888 23235 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:12.850980 23235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4"
format_stamp: "Formatted at 2026-08-09 16:37:10 on dist-test-slave-21b9"
server_key: "5c9887e03db8933bd26339ec41c81280"
server_key_iv: "fe714281f6eb5b7e1bb540f107fd7882"
server_key_version: "encryptionkey@0"
I20260809 16:37:12.851337 23235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:12.864699 23235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:12.866412 23235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:12.866536 23235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:12.870507 23235 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:38255
I20260809 16:37:12.870570 23351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:38255 every 8 connection(s)
I20260809 16:37:12.870894 23235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:12.872485 23352 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:37:12.873168 23352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap starting.
I20260809 16:37:12.880369 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 23235
I20260809 16:37:12.880630 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:43903
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=38761
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:12.882138 23352 log.cc:824] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:12.884270 23352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:12.884549 23352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4: Bootstrap complete.
I20260809 16:37:12.886675 23352 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:12.886889 23352 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Initialized, Role: FOLLOWER
I20260809 16:37:12.887033 23352 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:12.887111 23352 raft_consensus.cc:388] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:12.887162 23352 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:12.887208 23352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:37:12.888190 23352 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:12.888286 23352 leader_election.cc:302] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ded5a6fa9a8d4cbd9bc413bebd4d9cf4; no voters:
I20260809 16:37:12.888439 23352 leader_election.cc:288] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:37:12.888486 23357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:37:12.888621 23357 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [term 3 LEADER]: Becoming Leader. State: Replica: ded5a6fa9a8d4cbd9bc413bebd4d9cf4, State: Running, Role: LEADER
I20260809 16:37:12.888720 23352 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:12.888733 23357 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } }
I20260809 16:37:12.889073 23358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:12.889178 23358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:12.889415 23361 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded5a6fa9a8d4cbd9bc413bebd4d9cf4. Latest consensus state: current_term: 3 leader_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ded5a6fa9a8d4cbd9bc413bebd4d9cf4" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 38255 } } }
I20260809 16:37:12.889503 23370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:12.889532 23361 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded5a6fa9a8d4cbd9bc413bebd4d9cf4 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:12.890313 23370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:12.890425 23370 catalog_manager.cc:1259] Loaded cluster ID: a4b8f126917a410395324c08349b2590
I20260809 16:37:12.890491 23370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 16:37:12.890887 23370 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5342e479ad google::LogMessage::Fail() at ??:0
@ 0x7f5342e4a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5342e474de google::LogMessage::Flush() at ??:0
@ 0x7f5342e48a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f534ad680f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f534ad718b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5344d150d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5344d0a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5345b6e6db start_thread at ??:0
@ 0x7f534194561f clone at ??:0
W20260809 16:37:12.977710 23354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:12.977907 23354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:12.977931 23354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:12.979084 23354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:12.979125 23354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:12.979149 23354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:12.980638 23354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:43903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=38761
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:12.980862 23354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:12.981084 23354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:12.985867 23415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.985991 23418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:12.985867 23414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:12.986222 23354 server_base.cc:1034] running on GCE node
I20260809 16:37:12.986387 23354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:12.986572 23354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:12.987706 23354 hybrid_clock.cc:648] HybridClock initialized: now 1786293432987691 us; error 31 us; skew 500 ppm
Aug 09 16:37:12 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:12.990216 23354 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:12.990633 23354 webserver.cc:460] Webserver started at http://127.31.15.193:38761/ using document root <none> and password file <none>
I20260809 16:37:12.990847 23354 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:12.990896 23354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:12.992249 23354 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:12.992911 23428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:12.993062 23354 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:12.993144 23354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "849f607764fa4aeea15e11dafbf38ef1"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "084bb4ab40a70af56fb822b571cbb159"
server_key_iv: "84bde4f3e343fbfabf502a9469308db7"
server_key_version: "encryptionkey@0"
I20260809 16:37:12.993462 23354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:13.006317 23354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:13.008433 23354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:13.008567 23354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:13.008781 23354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:13.009208 23437 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:37:13.011165 23354 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:37:13.011207 23354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:37:13.011240 23354 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:37:13.011883 23354 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:37:13.011914 23354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:13.012010 23437 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap starting.
I20260809 16:37:13.018491 23354 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:43903
I20260809 16:37:13.018584 23625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:43903 every 8 connection(s)
I20260809 16:37:13.018893 23354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:13.023689 23437 log.cc:824] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:13.026403 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 23354
I20260809 16:37:13.026438 23437 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:13.026675 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:45425
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=37195
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:13.026746 23437 tablet_bootstrap.cc:492] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Bootstrap complete.
I20260809 16:37:13.027968 23437 ts_tablet_manager.cc:1397] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260809 16:37:13.029388 23437 raft_consensus.cc:348] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:13.029726 23437 raft_consensus.cc:729] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849f607764fa4aeea15e11dafbf38ef1, State: Initialized, Role: FOLLOWER
I20260809 16:37:13.029878 23437 consensus_queue.cc:260] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:13.030171 23437 ts_tablet_manager.cc:1428] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260809 16:37:13.075743 23628 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:37:13.019592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:36065)
0809 16:37:13.019779 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.019813 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:13.020241 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:13.075522 (+ 55281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
W20260809 16:37:13.076018 23626 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:38255 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:38255: Client connection negotiation failed: client connection to 127.31.15.254:38255: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:37:13.076359 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.076238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:52737)
0809 16:37:13.076291 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.076333 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:37:13.076666 23628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.076590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:48125)
0809 16:37:13.076626 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.076650 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:37:13.076757 23626 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:37:13.113317 23634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:13.113505 23634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:13.113526 23634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:13.114583 23634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:13.114617 23634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:13.114640 23634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:13.116055 23634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:45425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=37195
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:13.116255 23634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:13.116425 23634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:13.121035 23647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:13.121042 23651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:13.121050 23644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:13.121263 23634 server_base.cc:1034] running on GCE node
I20260809 16:37:13.121429 23634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:13.121635 23634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:13.122763 23634 hybrid_clock.cc:648] HybridClock initialized: now 1786293433122750 us; error 31 us; skew 500 ppm
Aug 09 16:37:13 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293433, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:13.125130 23634 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:13.125491 23634 webserver.cc:460] Webserver started at http://127.31.15.194:37195/ using document root <none> and password file <none>
I20260809 16:37:13.125669 23634 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:13.125708 23634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:13.126886 23634 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:37:13.127421 23659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.127562 23634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:13.127640 23634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "77c2639ae9f84599bedcedb42a60f819"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "bbc4eff3b5b8b4c333023c2a432da46c"
server_key_iv: "6d9d33e6d58b61ac9e568d7b12607949"
server_key_version: "encryptionkey@0"
I20260809 16:37:13.127951 23634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:13.157030 23634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:13.159255 23634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:13.159433 23634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:13.159662 23634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:13.159958 23634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:13.159991 23634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.160014 23634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:13.160037 23634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.165566 23634 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:45425
I20260809 16:37:13.165637 23840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:45425 every 8 connection(s)
I20260809 16:37:13.165947 23634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:37:13.166932 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.166548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:45035)
0809 16:37:13.166709 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.166785 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260809 16:37:13.167285 23841 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:38255 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:38255: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
I20260809 16:37:13.167516 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.167450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:45685)
0809 16:37:13.167477 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.167500 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260809 16:37:13.167706 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.167660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:32925)
0809 16:37:13.167679 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.167694 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 16:37:13.167776 23841 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:37:13.172994 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 23634
I20260809 16:37:13.173237 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:36331
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:38255
--webserver_port=35693
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:13.255515 23846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:13.255695 23846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:13.255717 23846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:13.256784 23846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:13.256817 23846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:13.256841 23846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:13.258234 23846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:36331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=35693
--tserver_master_addrs=127.31.15.254:38255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:13.258410 23846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:13.258597 23846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:13.263236 23854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:13.263247 23853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:13.263432 23846 server_base.cc:1034] running on GCE node
W20260809 16:37:13.263237 23856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:13.263669 23846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:13.263861 23846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:13.265007 23846 hybrid_clock.cc:648] HybridClock initialized: now 1786293433264996 us; error 29 us; skew 500 ppm
Aug 09 16:37:13 dist-test-slave-21b9 krb5kdc[21465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293433, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:13.267262 23846 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:13.267683 23846 webserver.cc:460] Webserver started at http://127.31.15.195:35693/ using document root <none> and password file <none>
I20260809 16:37:13.267866 23846 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:13.267910 23846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:13.269182 23846 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:13.269743 23871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.269883 23846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:37:13.269968 23846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "f9354705623e4cd2b8fbfeaa06c0994e"
format_stamp: "Formatted at 2026-08-09 16:37:11 on dist-test-slave-21b9"
server_key: "1ecd95d7975c4339f058d5eb9352753d"
server_key_iv: "98f2f08c88b873564e0cf2df309f05cb"
server_key_version: "encryptionkey@0"
I20260809 16:37:13.270268 23846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:13.296788 23846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:13.298512 23846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:13.298625 23846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:13.298812 23846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:13.299134 23846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:13.299166 23846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.299188 23846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:13.299212 23846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:13.304602 23846 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:36331
I20260809 16:37:13.304669 24075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:36331 every 8 connection(s)
I20260809 16:37:13.304953 23846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:37:13.305900 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.305561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:49331)
0809 16:37:13.305697 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.305767 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
W20260809 16:37:13.306241 24076 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:38255 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:38255: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
I20260809 16:37:13.306504 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.306408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:52989)
0809 16:37:13.306446 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.306483 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 16:37:13.306718 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:13.306671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:42267)
0809 16:37:13.306690 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:13.306707 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260809 16:37:13.306797 24076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:37:13.308692 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 23846
I20260809 16:37:14.077314 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.077059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:41401)
0809 16:37:14.077202 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:14.077240 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260809 16:37:14.168263 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.168059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:33417)
0809 16:37:14.168175 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:14.168209 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 16:37:14.307261 24085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.307065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:41217)
0809 16:37:14.307178 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:14.307210 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260809 16:37:14.759521 24086 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:14.759684 24086 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:14.760026 24086 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
Aug 09 16:37:14 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.195@KRBTEST.COM
Aug 09 16:37:14 dist-test-slave-21b9 krb5kdc[21465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293432, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.194@KRBTEST.COM
I20260809 16:37:14.769945 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.760194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:36331 (local address 127.31.15.193:52309)
0809 16:37:14.760332 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:14.760342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:14.760400 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:14.761175 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:14.761178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:14.761550 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:14.761742 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:14.761748 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.762820 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.762821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:14.763773 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:14.763776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.764322 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.764323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:14.764440 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:14.764946 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:14.764954 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:14.766383 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:14.768309 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768317 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:14.768319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:14.768640 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:14.768921 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:14.768923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:14.769046 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:14.769429 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:14.769437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:14.769570 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 16:37:14.769660 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:37:14.769843 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":67,"thread_start_us":75,"threads_started":1}
I20260809 16:37:14.769958 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.760290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:51457 (local address 127.31.15.194:45425)
0809 16:37:14.760446 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:14.760459 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:14.760486 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:37:14.760928 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:14.760930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:14.760969 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:14.761100 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:14.761857 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.762712 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.763849 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.764238 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.764340 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:14.766894 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:14.766898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:14.766898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:14.766907 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:14.768420 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768820 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:14.768822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:14.768822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:14.768912 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:14.769115 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:14.769116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:14.769116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:14.769380 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:14.769458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:14.769725 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 16:37:14.769838 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
I20260809 16:37:14.769990 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.760325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:52309 (local address 127.31.15.195:36331)
0809 16:37:14.760513 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:14.760521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:14.760549 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:37:14.760952 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:14.760954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:14.761001 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:14.761120 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:14.761857 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.762733 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.763846 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.764240 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.764337 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:14.766482 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:14.766487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:14.766488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:14.766499 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:14.768230 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768735 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:14.768739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:14.768739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:14.768859 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:14.769109 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:14.769110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:14.769111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:14.769360 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:14.769427 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:14.769735 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 16:37:14.769862 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":56,"threads_started":1}
I20260809 16:37:14.770033 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:14.760192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:45425 (local address 127.31.15.193:51457)
0809 16:37:14.760330 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:14.760342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:14.760400 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:14.761178 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:14.761179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:14.761541 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:14.761742 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:14.761747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.762810 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.762811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:14.763774 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:14.763776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:14.764319 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:14.764320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:14.764441 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:14.764996 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:14.765000 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:14.766793 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:14.768514 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:14.768515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:14.768741 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:14.768979 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:14.768980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:14.768980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:14.769055 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:14.769474 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:14.769475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:14.769571 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 16:37:14.769661 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:37:14.769915 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":159,"thread_start_us":50,"threads_started":1}
W20260809 16:37:14.771402 23433 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
W20260809 16:37:14.771481 23434 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:14.771539 23434 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1; no voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e
I20260809 16:37:14.771636 24086 raft_consensus.cc:2726] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:37:15.077925 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:15.077775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:59095)
0809 16:37:15.077845 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:15.077883 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 16:37:15.168929 24096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:15.168691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:33027)
0809 16:37:15.168823 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:15.168876 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260809 16:37:15.307931 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:15.307723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:36349)
0809 16:37:15.307856 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:15.307890 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260809 16:37:16.078649 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:16.078384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:43113)
0809 16:37:16.078545 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:16.078594 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":75,"threads_started":1}
I20260809 16:37:16.169656 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:16.169418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:39003)
0809 16:37:16.169574 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:16.169612 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260809 16:37:16.308580 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:16.308376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:37355)
0809 16:37:16.308501 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:16.308538 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260809 16:37:16.850592 24109 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:16.850696 24109 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:16.850858 24109 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
W20260809 16:37:16.851264 23434 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
W20260809 16:37:16.851389 23433 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:16.851454 23433 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1; no voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e
I20260809 16:37:16.851539 24109 raft_consensus.cc:2726] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:37:17.079375 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:17.079085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:58297)
0809 16:37:17.079240 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:17.079300 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260809 16:37:17.170336 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:17.170098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:49879)
0809 16:37:17.170230 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:17.170270 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260809 16:37:17.309194 24113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:17.308984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:50927)
0809 16:37:17.309106 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:17.309155 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260809 16:37:18.080071 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:18.079834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:43647)
0809 16:37:18.079968 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:18.080008 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260809 16:37:18.170940 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:18.170731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:49327)
0809 16:37:18.170860 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:18.170899 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260809 16:37:18.309782 24117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:18.309573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:35893)
0809 16:37:18.309705 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:18.309740 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260809 16:37:19.080818 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:19.080563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:54507)
0809 16:37:19.080728 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:19.080768 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260809 16:37:19.137713 24121 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:19.137830 24121 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:19.138015 24121 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
W20260809 16:37:19.138391 23433 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
W20260809 16:37:19.138494 23434 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:19.138567 23434 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1; no voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e
I20260809 16:37:19.138636 24121 raft_consensus.cc:2726] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:37:19.171567 24123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:19.171339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:41153)
0809 16:37:19.171488 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:19.171524 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260809 16:37:19.310380 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:19.310179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:40047)
0809 16:37:19.310307 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:19.310340 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
I20260809 16:37:20.081427 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:20.081219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:38399)
0809 16:37:20.081346 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:20.081378 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260809 16:37:20.172271 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:20.172039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:49951)
0809 16:37:20.172187 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:20.172226 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260809 16:37:20.311018 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:20.310810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:49245)
0809 16:37:20.310944 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:20.310978 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260809 16:37:21.082195 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:21.081947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:59245)
0809 16:37:21.082109 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:21.082144 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260809 16:37:21.172973 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:21.172779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:47949)
0809 16:37:21.172897 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:21.172931 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260809 16:37:21.311699 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:21.311487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:48515)
0809 16:37:21.311622 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:21.311658 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260809 16:37:22.082969 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:22.082710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:54745)
0809 16:37:22.082876 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:22.082909 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260809 16:37:22.173650 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:22.173421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:49371)
0809 16:37:22.173566 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:22.173605 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 16:37:22.312309 24146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:22.312093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:44975)
0809 16:37:22.312226 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:22.312265 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260809 16:37:23.054589 24148 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:23.054700 24148 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:23.054879 24148 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
W20260809 16:37:23.055315 23434 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
W20260809 16:37:23.055418 23433 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:23.055474 23433 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1; no voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e
I20260809 16:37:23.055547 24148 raft_consensus.cc:2726] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:37:23.083784 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:23.083522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:50079)
0809 16:37:23.083684 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:23.083719 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260809 16:37:23.174345 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:23.174082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:41475)
0809 16:37:23.174241 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:23.174283 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260809 16:37:23.312999 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:23.312750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:41455)
0809 16:37:23.312893 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:23.312934 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260809 16:37:24.084507 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:24.084260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:50757)
0809 16:37:24.084395 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:24.084444 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260809 16:37:24.175053 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:24.174809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:56483)
0809 16:37:24.174956 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:24.174999 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
I20260809 16:37:24.313623 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:24.313426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:59423)
0809 16:37:24.313543 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:24.313577 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
I20260809 16:37:25.085152 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:25.084924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:42999)
0809 16:37:25.085060 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:25.085101 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260809 16:37:25.175812 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:25.175562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:33185)
0809 16:37:25.175714 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:25.175763 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":60,"threads_started":1}
I20260809 16:37:25.314225 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:25.314022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:48279)
0809 16:37:25.314144 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:25.314181 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260809 16:37:25.679641 24164 raft_consensus.cc:482] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:25.679734 24164 raft_consensus.cc:504] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77c2639ae9f84599bedcedb42a60f819" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 45425 } } peers { permanent_uuid: "f9354705623e4cd2b8fbfeaa06c0994e" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 36331 } } peers { permanent_uuid: "849f607764fa4aeea15e11dafbf38ef1" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 43903 } }
I20260809 16:37:25.679880 24164 leader_election.cc:288] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425), f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331)
W20260809 16:37:25.680253 23434 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 77c2639ae9f84599bedcedb42a60f819 (127.31.15.194:45425): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
W20260809 16:37:25.680334 23433 leader_election.cc:341] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9354705623e4cd2b8fbfeaa06c0994e (127.31.15.195:36331): Not found: Tablet not found: 68628d35a2b5449a89cac892c33689f4
I20260809 16:37:25.680380 23433 leader_election.cc:302] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 849f607764fa4aeea15e11dafbf38ef1; no voters: 77c2639ae9f84599bedcedb42a60f819, f9354705623e4cd2b8fbfeaa06c0994e
I20260809 16:37:25.680474 24164 raft_consensus.cc:2726] T 68628d35a2b5449a89cac892c33689f4 P 849f607764fa4aeea15e11dafbf38ef1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:37:26.085899 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:26.085671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:50081)
0809 16:37:26.085813 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:26.085851 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260809 16:37:26.176417 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:26.176223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:51257)
0809 16:37:26.176345 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:26.176378 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
I20260809 16:37:26.314853 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:26.314650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:35199)
0809 16:37:26.314771 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:26.314808 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
I20260809 16:37:27.086699 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:27.086435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:55227)
0809 16:37:27.086591 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:27.086647 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 16:37:27.177065 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:27.176829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:48265)
0809 16:37:27.176979 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:27.177016 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":61,"threads_started":1}
I20260809 16:37:27.315579 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:27.315358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:53713)
0809 16:37:27.315486 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:27.315536 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260809 16:37:28.087466 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:28.087213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.193:47817)
0809 16:37:28.087382 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:28.087418 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260809 16:37:28.177814 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:28.177567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.194:55753)
0809 16:37:28.177720 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:28.177763 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260809 16:37:28.312932 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 23354
I20260809 16:37:28.321409 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:28.321145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:38255 (local address 127.31.15.195:57805)
0809 16:37:28.321300 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:28.321340 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260809 16:37:28.325613 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 23634
I20260809 16:37:28.337507 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 23846
2026-08-09T16:37:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18802 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:37:28 dist-test-slave-21b9 krb5kdc[24180](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:37:28 dist-test-slave-21b9 krb5kdc[24180](info): set up 2 sockets
Aug 09 16:37:28 dist-test-slave-21b9 krb5kdc[24180](info): commencing operation
krb5kdc: starting...
W20260809 16:37:29.504822 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:37:29 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:37:29.625694 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:32839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:29.712548 24195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:29.712745 24195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:29.712769 24195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:29.713886 24195 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:37:29.713927 24195 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:37:29.713939 24195 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:37:29.713949 24195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:29.713961 24195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:29.713985 24195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:29.714008 24195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:29.714025 24195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:29.715507 24195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:32839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:29.715723 24195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:29.715946 24195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:29.721655 24204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:29.721695 24207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:29.721648 24203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:29.724799 24195 server_base.cc:1034] running on GCE node
I20260809 16:37:29.725119 24195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:29.725466 24195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:29.726603 24195 hybrid_clock.cc:648] HybridClock initialized: now 1786293449726585 us; error 37 us; skew 500 ppm
Aug 09 16:37:29 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293449, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:29.729193 24195 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:29.729616 24195 webserver.cc:460] Webserver started at http://127.31.15.254:39119/ using document root <none> and password file <none>
I20260809 16:37:29.729851 24195 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:29.729908 24195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:29.730043 24195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:29.731094 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "1f480fd241534f1f81686e6ff24d7321"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "6dc56cdcaa74c8df21cd8fd84e4a60d6"
server_key_iv: "669d9a1c906992f582fa673244528fbf"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.731462 24195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "1f480fd241534f1f81686e6ff24d7321"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "6dc56cdcaa74c8df21cd8fd84e4a60d6"
server_key_iv: "669d9a1c906992f582fa673244528fbf"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.732715 24195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:37:29.733415 24224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:29.733595 24195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:29.733681 24195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "1f480fd241534f1f81686e6ff24d7321"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "6dc56cdcaa74c8df21cd8fd84e4a60d6"
server_key_iv: "669d9a1c906992f582fa673244528fbf"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.733748 24195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:29.746711 24195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:29.748418 24195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:29.748540 24195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:29.752426 24195 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:32839
I20260809 16:37:29.752460 24300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:32839 every 8 connection(s)
I20260809 16:37:29.752780 24195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:29.753633 24301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:29.755861 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap starting.
I20260809 16:37:29.756402 24301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:29.756649 24301 log.cc:824] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:29.757262 24301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: No bootstrap required, opened a new log
I20260809 16:37:29.758808 24301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:29.758940 24301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:29.758961 24301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Initialized, Role: FOLLOWER
I20260809 16:37:29.759070 24301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:29.759150 24301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:29.759178 24301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:29.759207 24301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:29.759714 24301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:29.759799 24301 leader_election.cc:302] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f480fd241534f1f81686e6ff24d7321; no voters:
I20260809 16:37:29.759928 24301 leader_election.cc:288] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:37:29.759987 24308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:29.760165 24301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:29.760171 24308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 LEADER]: Becoming Leader. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Running, Role: LEADER
I20260809 16:37:29.760332 24308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:29.760723 24309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:37:29.760828 24309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:29.760703 24310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f480fd241534f1f81686e6ff24d7321. Latest consensus state: current_term: 1 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:37:29.760901 24310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:29.761039 24322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:29.761168 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 24195
I20260809 16:37:29.761266 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:37:29.761480 31807 external_mini_cluster.cc:1428] Setting key 47ef46f6805ee2f50be7a5f264604afc
I20260809 16:37:29.761608 24322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:29.763404 24322 catalog_manager.cc:1347] Generated new cluster ID: 5bddf68743fc428bbbd3cced235a78e1
I20260809 16:37:29.763469 24322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:37:29 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:29.771085 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:29.762154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.31.15.254:32839)
0809 16:37:29.762403 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:29.762412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:29.762444 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:37:29.762570 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:29.762577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:29.762631 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:29.762775 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:29.763752 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.764646 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.765365 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.765800 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.765894 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:29.768130 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:29.768134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:29.768135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:29.768147 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:29.769664 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:29.769989 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:29.769992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:29.769992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:29.770088 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:29.770272 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:29.770273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:29.770273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:29.770504 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:29.770636 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:29.770751 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 16:37:29.770882 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":70,"threads_started":1}
I20260809 16:37:29.799312 24322 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:37:29.799836 24322 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:29.804378 24322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Generated new TSK 0
I20260809 16:37:29.804543 24322 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:37:29.836371 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:29.940210 24346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:29.940407 24346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:29.940429 24346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:29.941604 24346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:29.941648 24346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:29.941673 24346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:29.943076 24346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:29.943287 24346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:29.943504 24346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:29.948242 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:29.948246 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:29.948239 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:29.948485 24346 server_base.cc:1034] running on GCE node
I20260809 16:37:29.948661 24346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:29.948843 24346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:29.949978 24346 hybrid_clock.cc:648] HybridClock initialized: now 1786293449949966 us; error 27 us; skew 500 ppm
Aug 09 16:37:29 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293449, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:29.952458 24346 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:29.952857 24346 webserver.cc:460] Webserver started at http://127.31.15.193:44315/ using document root <none> and password file <none>
I20260809 16:37:29.953083 24346 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:29.953132 24346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:29.953256 24346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:29.954237 24346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "eb320913384c46359f12a42d710c1063"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "26234043cc2d7b21ccd963ea2817b471"
server_key_iv: "3115063ec45929dd64fd418137256fb7"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.954551 24346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "eb320913384c46359f12a42d710c1063"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "26234043cc2d7b21ccd963ea2817b471"
server_key_iv: "3115063ec45929dd64fd418137256fb7"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.955834 24346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 16:37:29.956475 24368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:29.956616 24346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:29.956697 24346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "eb320913384c46359f12a42d710c1063"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "26234043cc2d7b21ccd963ea2817b471"
server_key_iv: "3115063ec45929dd64fd418137256fb7"
server_key_version: "encryptionkey@0"
I20260809 16:37:29.956754 24346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:29.979070 24346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:29.981073 24346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:29.981216 24346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:29.981451 24346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:29.981777 24346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:29.981820 24346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:29.981850 24346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:29.981870 24346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:29.987183 24346 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:41235
I20260809 16:37:29.987248 24558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:41235 every 8 connection(s)
I20260809 16:37:29.987553 24346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:29.992004 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 24346
I20260809 16:37:29.992105 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:37:29.992301 31807 external_mini_cluster.cc:1428] Setting key 0c096a69e607510be6f349c0023d9e5b
Aug 09 16:37:29 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293449, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:29.996459 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:29.988240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39193 (local address 127.31.15.254:32839)
0809 16:37:29.988284 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:29.988286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:29.988478 (+ 192us) server_negotiation.cc:407] Connection header received
0809 16:37:29.988952 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:29.988960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:29.988975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:29.989029 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:29.989909 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.990420 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.991400 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.991608 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.991662 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:29.993782 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:29.993784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:29.993784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:29.993794 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:29.995144 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:29.995579 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:29.995580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:29.995580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:29.995645 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:29.995864 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:29.995865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:29.995865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:29.996008 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:29.996083 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:29.996292 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:37:29.996374 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:37:29.996523 24562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:29.988190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:39193)
0809 16:37:29.988336 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:29.988377 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:29.988843 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:29.989088 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:29.989091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:29.989634 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:29.989823 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:29.989827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.990489 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.990490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:29.991329 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:29.991332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:29.991668 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:29.991669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:29.991760 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:29.992253 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:29.992261 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:29.993694 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:29.995241 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:29.995248 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:29.995250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:29.995502 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:29.995706 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:29.995706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:29.995707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:29.995812 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:29.996071 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:29.996074 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:29.996179 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 16:37:29.996254 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:37:29.996368 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260809 16:37:29.996982 24560 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:29.997074 24560 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:29.997232 24560 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:29.997792 24250 ts_manager.cc:194] Registered new tserver with Master: eb320913384c46359f12a42d710c1063 (127.31.15.193:41235)
I20260809 16:37:29.998674 24250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:39193
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:37:30.019016 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:30.102293 24567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:30.102512 24567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:30.102536 24567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:30.103746 24567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:30.103790 24567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:30.103819 24567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:30.105300 24567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:30.105535 24567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:30.105743 24567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:30.110666 24577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:30.110666 24573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:30.110652 24582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:30.110930 24567 server_base.cc:1034] running on GCE node
I20260809 16:37:30.111107 24567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:30.111340 24567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:30.112488 24567 hybrid_clock.cc:648] HybridClock initialized: now 1786293450112477 us; error 28 us; skew 500 ppm
Aug 09 16:37:30 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293450, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:30.117781 24567 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:30.118201 24567 webserver.cc:460] Webserver started at http://127.31.15.194:39961/ using document root <none> and password file <none>
I20260809 16:37:30.118407 24567 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:30.118450 24567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:30.118582 24567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:30.119640 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "15f341b4f0584a16a90d5c506ed9611e"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "0142b50f210c121e17f2d6a91643ea9e"
server_key_iv: "745496b8b2ff6d6f163ce40c2ad24775"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.119964 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "15f341b4f0584a16a90d5c506ed9611e"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "0142b50f210c121e17f2d6a91643ea9e"
server_key_iv: "745496b8b2ff6d6f163ce40c2ad24775"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.121201 24567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:30.121873 24590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.122025 24567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.122103 24567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "15f341b4f0584a16a90d5c506ed9611e"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "0142b50f210c121e17f2d6a91643ea9e"
server_key_iv: "745496b8b2ff6d6f163ce40c2ad24775"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.122167 24567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:30.140084 24567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:30.141947 24567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:30.142081 24567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:30.142290 24567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:30.142604 24567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:30.142647 24567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.142675 24567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:30.142692 24567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.148228 24567 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33001
I20260809 16:37:30.148298 24763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33001 every 8 connection(s)
I20260809 16:37:30.148577 24567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:37:30 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293450, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:30.154676 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 24567
I20260809 16:37:30.154768 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:37:30.154953 31807 external_mini_cluster.cc:1428] Setting key 2b689f250b2638343dd8fc833c69c0b4
I20260809 16:37:30.157203 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.149233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:59117 (local address 127.31.15.254:32839)
0809 16:37:30.149330 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.149333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.149476 (+ 143us) server_negotiation.cc:407] Connection header received
0809 16:37:30.149913 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.149914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.149925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.149974 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:30.150858 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.151397 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.152313 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.152489 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.152538 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.154447 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:30.154449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:30.154450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:30.154457 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:30.156016 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.156361 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.156362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.156362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.156428 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.156630 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.156631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.156631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.156754 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:30.156816 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.157025 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.157116 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 16:37:30.157266 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.149201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:59117)
0809 16:37:30.149358 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.149394 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.149837 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.150035 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.150039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.150580 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:30.150763 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.150768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.151472 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.151472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.152251 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.152254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.152549 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.152549 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.152641 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.153047 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:30.153055 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:30.154365 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:30.156084 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:30.156091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:30.156093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:30.156293 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:30.156482 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:30.156482 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:30.156483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:30.156577 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:30.156808 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:30.156811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:30.156914 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.156988 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.157109 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260809 16:37:30.157809 24764 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:30.157912 24764 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:30.158087 24764 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:30.158437 24250 ts_manager.cc:194] Registered new tserver with Master: 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001)
I20260809 16:37:30.158792 24250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:59117
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:37:30.180058 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:30.260076 24772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:30.260309 24772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:30.260335 24772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:30.261462 24772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:30.261507 24772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:30.261533 24772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:30.263052 24772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:30.263257 24772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:30.263494 24772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:30.267966 24781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:30.267998 24779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:30.267968 24778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:30.268291 24772 server_base.cc:1034] running on GCE node
I20260809 16:37:30.268496 24772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:30.268712 24772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:30.269845 24772 hybrid_clock.cc:648] HybridClock initialized: now 1786293450269832 us; error 31 us; skew 500 ppm
Aug 09 16:37:30 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293450, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:30.272452 24772 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:30.272846 24772 webserver.cc:460] Webserver started at http://127.31.15.195:35751/ using document root <none> and password file <none>
I20260809 16:37:30.273036 24772 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:30.273077 24772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:30.273166 24772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:37:30.274078 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "d6857373d6f0366923838d515dc818e2"
server_key_iv: "d4651c516cf78ae216dcf14d7d35e87b"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.274398 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "d6857373d6f0366923838d515dc818e2"
server_key_iv: "d4651c516cf78ae216dcf14d7d35e87b"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.275599 24772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:37:30.276207 24789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.276348 24772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:37:30.276448 24772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "d6857373d6f0366923838d515dc818e2"
server_key_iv: "d4651c516cf78ae216dcf14d7d35e87b"
server_key_version: "encryptionkey@0"
I20260809 16:37:30.276508 24772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:30.285496 24772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:30.287231 24772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:30.287424 24772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:30.287626 24772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:30.287895 24772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:30.287930 24772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.287957 24772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:30.287974 24772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:30.293198 24772 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:42517
I20260809 16:37:30.293268 24969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:42517 every 8 connection(s)
I20260809 16:37:30.293526 24772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:37:30.294660 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 24772
I20260809 16:37:30.294752 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:37:30.294956 31807 external_mini_cluster.cc:1428] Setting key fcaf5959fcda1c4309a9a77b77e232c8
Aug 09 16:37:30 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293450, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:30.301968 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.294182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:57437 (local address 127.31.15.254:32839)
0809 16:37:30.294239 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.294241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.294399 (+ 158us) server_negotiation.cc:407] Connection header received
0809 16:37:30.294907 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.294910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.294925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.295010 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:30.295898 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.296398 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.297295 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.297488 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.297543 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.299406 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:30.299408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:30.299408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:30.299417 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:30.300813 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.301187 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.301189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.301189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.301249 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.301464 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.301465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.301465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.301589 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:30.301652 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.301837 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.301905 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:37:30.302107 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.294134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:57437)
0809 16:37:30.294280 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.294320 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.294804 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.295082 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.295086 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.295643 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:30.295823 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.295827 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.296469 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.296470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.297226 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.297228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.297552 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.297553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.297642 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.298045 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:30.298053 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:30.299332 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:30.300885 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:30.300891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:30.300892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:30.301092 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:30.301308 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:30.301309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:30.301309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:30.301390 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:30.301646 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:30.301649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:30.301746 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.301815 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.301929 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260809 16:37:30.302564 24971 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:30.302642 24971 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:30.302778 24971 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:30.303086 24250 ts_manager.cc:194] Registered new tserver with Master: 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
I20260809 16:37:30.303506 24250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:57437
I20260809 16:37:30.306504 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:30.312747 24339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.307807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.31.15.254:32839)
0809 16:37:30.307839 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.307841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.307932 (+ 91us) server_negotiation.cc:407] Connection header received
0809 16:37:30.307979 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.307979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.307986 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.308029 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:30.308483 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.308922 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.309570 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.309745 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.309786 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.310734 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:30.310735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:30.310735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:30.310741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:30.311918 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.312205 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.312206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.312206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.312250 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:30.312380 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:30.312381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:30.312381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:30.312504 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:30.312574 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.312640 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.312692 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:37:30.314101 24250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:37:30.315621 24250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:37:30.321291 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.317672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:42517 (local address 127.0.0.1:53690)
0809 16:37:30.317900 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.317909 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.317954 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.318238 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.318241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.318243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:30.318299 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.318300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319211 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.319212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.320109 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.320111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.321013 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.321014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.321098 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.321113 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.321170 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.321220 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":71,"threads_started":1}
I20260809 16:37:30.321301 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.317950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53690 (local address 127.31.15.195:42517)
0809 16:37:30.318096 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.318100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.318107 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:37:30.318135 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.318136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.318147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.318211 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:30.318363 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319120 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.320177 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.320865 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.320931 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.320980 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.321196 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.321234 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260809 16:37:30.321858 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.317625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.0.0.1:57528)
0809 16:37:30.317789 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.317828 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.317893 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.318587 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.318588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.318589 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:30.318617 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.318618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319968 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.319969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.320846 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.320849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.321640 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.321641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.321701 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.321705 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.321761 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.321796 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":47,"threads_started":1}
I20260809 16:37:30.321903 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.317757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.31.15.193:41235)
0809 16:37:30.318433 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.318437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.318446 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:37:30.318483 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.318485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.318498 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.318686 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:30.318763 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319852 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.320925 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.321568 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.321640 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.321694 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.321784 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.321826 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":85,"threads_started":1}
I20260809 16:37:30.322153 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.317802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33001 (local address 127.0.0.1:53198)
0809 16:37:30.318242 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.318252 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.318307 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.318543 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.318544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.318546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:30.318595 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.318597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319906 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.319907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.321065 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.321067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.321971 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.321972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.322036 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.322041 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.322064 (+ 23us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.322091 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":52,"thread_start_us":55,"threads_started":1}
I20260809 16:37:30.322239 24869 tablet_service.cc:1467] Processing CreateTablet for tablet 582a7757f9114d2585bcec3b676f975b (DEFAULT_TABLE table=test-table [id=0e22eea875f344c0a88719e073711d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:30.322458 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.318098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.31.15.194:33001)
0809 16:37:30.318250 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.318254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.318296 (+ 42us) server_negotiation.cc:407] Connection header received
0809 16:37:30.318366 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.318368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.318391 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.318464 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:30.318671 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.319788 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.321150 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.321859 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.322266 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.322325 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.322347 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.322381 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260809 16:37:30.322580 24869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582a7757f9114d2585bcec3b676f975b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:30.322793 24444 tablet_service.cc:1467] Processing CreateTablet for tablet 582a7757f9114d2585bcec3b676f975b (DEFAULT_TABLE table=test-table [id=0e22eea875f344c0a88719e073711d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:30.323083 24444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582a7757f9114d2585bcec3b676f975b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:30.325212 25004 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Bootstrap starting.
I20260809 16:37:30.325635 25005 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Bootstrap starting.
I20260809 16:37:30.325771 24659 tablet_service.cc:1467] Processing CreateTablet for tablet 582a7757f9114d2585bcec3b676f975b (DEFAULT_TABLE table=test-table [id=0e22eea875f344c0a88719e073711d27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:30.326095 25004 tablet_bootstrap.cc:654] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:30.326107 24659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 582a7757f9114d2585bcec3b676f975b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:30.326278 25005 tablet_bootstrap.cc:654] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:30.326640 25004 log.cc:824] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:30.326773 25005 log.cc:824] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:30.327503 25005 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: No bootstrap required, opened a new log
I20260809 16:37:30.327605 25005 ts_tablet_manager.cc:1397] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:37:30.327852 25004 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: No bootstrap required, opened a new log
I20260809 16:37:30.327920 25004 ts_tablet_manager.cc:1397] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 16:37:30.328882 25008 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Bootstrap starting.
I20260809 16:37:30.329416 25008 tablet_bootstrap.cc:654] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:30.329491 25005 raft_consensus.cc:348] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.329654 25005 raft_consensus.cc:374] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:30.329687 25005 raft_consensus.cc:729] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb320913384c46359f12a42d710c1063, State: Initialized, Role: FOLLOWER
I20260809 16:37:30.329751 25004 raft_consensus.cc:348] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.329833 25005 consensus_queue.cc:260] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.329890 25004 raft_consensus.cc:374] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:30.329885 25008 log.cc:824] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:30.329926 25004 raft_consensus.cc:729] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a12282ec58a4e9eb724fa32f2ee2321, State: Initialized, Role: FOLLOWER
I20260809 16:37:30.330031 25004 consensus_queue.cc:260] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.330116 25005 ts_tablet_manager.cc:1428] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:37:30.330147 24560 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
I20260809 16:37:30.330278 25004 ts_tablet_manager.cc:1428] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:30.330298 24971 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
I20260809 16:37:30.330598 25008 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: No bootstrap required, opened a new log
I20260809 16:37:30.330673 25008 ts_tablet_manager.cc:1397] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:30.332165 25008 raft_consensus.cc:348] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.332273 25008 raft_consensus.cc:374] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:30.332293 25008 raft_consensus.cc:729] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f341b4f0584a16a90d5c506ed9611e, State: Initialized, Role: FOLLOWER
I20260809 16:37:30.332365 25008 consensus_queue.cc:260] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.332553 25008 ts_tablet_manager.cc:1428] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:30.332631 24764 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
W20260809 16:37:30.399109 24765 tablet.cc:2369] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:30.488286 24561 tablet.cc:2369] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:30.544080 24972 tablet.cc:2369] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:30.594949 25013 raft_consensus.cc:482] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:30.595080 25013 raft_consensus.cc:504] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.595484 25013 leader_election.cc:288] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb320913384c46359f12a42d710c1063 (127.31.15.193:41235), 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001)
I20260809 16:37:30.599217 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.595787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:56425 (local address 127.31.15.193:41235)
0809 16:37:30.595846 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.595848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.595859 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:30.595917 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.595918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.595928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.595986 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:30.596140 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.596911 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598285 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.598863 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598938 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.598946 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.599104 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.599150 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:37:30.599238 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.595906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:58403 (local address 127.31.15.194:33001)
0809 16:37:30.595963 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.595964 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.596072 (+ 108us) server_negotiation.cc:407] Connection header received
0809 16:37:30.596133 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.596134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.596143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.596195 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:30.596329 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.597004 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598293 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.598862 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598916 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.598926 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.599132 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.599180 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:37:30.599227 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.595716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.31.15.195:56425)
0809 16:37:30.595773 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.595784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.595849 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.595999 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.596001 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.596003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:30.596076 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.596078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.596988 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.596989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.598199 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.598201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.598941 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.599025 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.599028 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.599072 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.599118 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":31}
I20260809 16:37:30.599238 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.595826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33001 (local address 127.31.15.195:58403)
0809 16:37:30.595989 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.595999 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:30.596071 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:30.596215 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:30.596216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:30.596217 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:30.596270 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.596271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.597069 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.597070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.598189 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:30.598193 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.598986 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.598986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:30.599055 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.599058 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:30.599107 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:37:30.599146 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":102,"threads_started":1}
I20260809 16:37:30.599602 24691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582a7757f9114d2585bcec3b676f975b" candidate_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f341b4f0584a16a90d5c506ed9611e" is_pre_election: true
I20260809 16:37:30.599685 24480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582a7757f9114d2585bcec3b676f975b" candidate_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb320913384c46359f12a42d710c1063" is_pre_election: true
I20260809 16:37:30.599745 24691 raft_consensus.cc:2454] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a12282ec58a4e9eb724fa32f2ee2321 in term 0.
I20260809 16:37:30.599840 24480 raft_consensus.cc:2454] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a12282ec58a4e9eb724fa32f2ee2321 in term 0.
I20260809 16:37:30.599959 24793 leader_election.cc:302] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321; no voters:
I20260809 16:37:30.600113 25013 raft_consensus.cc:2781] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:30.600185 25013 raft_consensus.cc:482] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:30.600214 25013 raft_consensus.cc:3037] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:30.600914 25013 raft_consensus.cc:504] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.601029 25013 leader_election.cc:288] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [CANDIDATE]: Term 1 election: Requested vote from peers eb320913384c46359f12a42d710c1063 (127.31.15.193:41235), 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001)
I20260809 16:37:30.601205 24480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582a7757f9114d2585bcec3b676f975b" candidate_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb320913384c46359f12a42d710c1063"
I20260809 16:37:30.601246 24691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "582a7757f9114d2585bcec3b676f975b" candidate_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15f341b4f0584a16a90d5c506ed9611e"
I20260809 16:37:30.601296 24480 raft_consensus.cc:3037] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:30.601308 24691 raft_consensus.cc:3037] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:30.602190 24691 raft_consensus.cc:2454] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a12282ec58a4e9eb724fa32f2ee2321 in term 1.
I20260809 16:37:30.602190 24480 raft_consensus.cc:2454] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a12282ec58a4e9eb724fa32f2ee2321 in term 1.
I20260809 16:37:30.602352 24793 leader_election.cc:302] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a12282ec58a4e9eb724fa32f2ee2321, eb320913384c46359f12a42d710c1063; no voters:
I20260809 16:37:30.602443 25013 raft_consensus.cc:2781] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:30.602594 25013 raft_consensus.cc:686] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 LEADER]: Becoming Leader. State: Replica: 7a12282ec58a4e9eb724fa32f2ee2321, State: Running, Role: LEADER
I20260809 16:37:30.602689 25013 consensus_queue.cc:237] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:30.603502 24250 catalog_manager.cc:5495] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195). New cstate: current_term: 1 leader_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } health_report { overall_health: HEALTHY } } }
I20260809 16:37:30.672119 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.670026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.31.15.195:42517)
0809 16:37:30.670078 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:30.670080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:30.670089 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:37:30.670129 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:30.670130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:30.670149 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:30.670208 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:30.670304 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.670758 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.671510 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:30.671684 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:30.671728 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:30.671828 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:30.671925 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:30.671980 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:30.671997 (+ 17us) server_negotiation.cc:299] Negotiation successful
0809 16:37:30.672013 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:37:30.673908 24691 raft_consensus.cc:1261] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Refusing update from remote peer 7a12282ec58a4e9eb724fa32f2ee2321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:30.673904 24480 raft_consensus.cc:1261] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Refusing update from remote peer 7a12282ec58a4e9eb724fa32f2ee2321: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:30.674244 25020 consensus_queue.cc:1035] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:30.674364 25013 consensus_queue.cc:1035] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:30.676832 25026 mvcc.cc:205] Tried to move back new op lower bound from 7316657973957484544 to 7316657973669199872. Current Snapshot: MvccSnapshot[applied={T|T < 7316657973957484544}]
I20260809 16:37:30.676996 25027 mvcc.cc:205] Tried to move back new op lower bound from 7316657973957484544 to 7316657973669199872. Current Snapshot: MvccSnapshot[applied={T|T < 7316657973957484544}]
I20260809 16:37:30.678791 24250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:30.678889 24250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:30.679636 24250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Sending DeleteTablet for 3 replicas of tablet 582a7757f9114d2585bcec3b676f975b
I20260809 16:37:30.679982 24444 tablet_service.cc:1514] Processing DeleteTablet for tablet 582a7757f9114d2585bcec3b676f975b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:30 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:57528
I20260809 16:37:30.680011 24659 tablet_service.cc:1514] Processing DeleteTablet for tablet 582a7757f9114d2585bcec3b676f975b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:30 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53198
I20260809 16:37:30.680033 24869 tablet_service.cc:1514] Processing DeleteTablet for tablet 582a7757f9114d2585bcec3b676f975b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:30 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53690
I20260809 16:37:30.680227 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 24346
I20260809 16:37:30.680248 25038 tablet_replica.cc:330] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: stopping tablet replica
I20260809 16:37:30.689448 25039 tablet_replica.cc:330] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: stopping tablet replica
I20260809 16:37:30.689450 25041 tablet_replica.cc:330] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: stopping tablet replica
I20260809 16:37:30.689564 25041 raft_consensus.cc:2229] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:30.689564 25039 raft_consensus.cc:2229] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:37:30.689679 25039 raft_consensus.cc:2258] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:30.689683 25041 raft_consensus.cc:2258] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:30.691874 25041 ts_tablet_manager.cc:1905] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:30.693406 25041 ts_tablet_manager.cc:1918] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:30.693461 25041 log.cc:1196] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/582a7757f9114d2585bcec3b676f975b
I20260809 16:37:30.693670 25041 ts_tablet_manager.cc:1939] T 582a7757f9114d2585bcec3b676f975b P 15f341b4f0584a16a90d5c506ed9611e: Deleting consensus metadata
I20260809 16:37:30.694047 24232 catalog_manager.cc:4841] TS 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): tablet 582a7757f9114d2585bcec3b676f975b (table test-table [id=0e22eea875f344c0a88719e073711d27]) successfully deleted
I20260809 16:37:30.698458 25039 ts_tablet_manager.cc:1905] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:30.700368 25039 ts_tablet_manager.cc:1918] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:30.700521 25039 log.cc:1196] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/582a7757f9114d2585bcec3b676f975b
I20260809 16:37:30.700866 25039 ts_tablet_manager.cc:1939] T 582a7757f9114d2585bcec3b676f975b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting consensus metadata
I20260809 16:37:30.701339 24230 catalog_manager.cc:4841] TS 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): tablet 582a7757f9114d2585bcec3b676f975b (table test-table [id=0e22eea875f344c0a88719e073711d27]) successfully deleted
W20260809 16:37:30.705178 24232 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:41235 (error 108)
I20260809 16:37:30.705719 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:30.705582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.0.0.1:57532)
0809 16:37:30.705642 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:30.705683 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:41235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 16:37:30.705767 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 24567
W20260809 16:37:30.716241 24232 catalog_manager.cc:4584] TS eb320913384c46359f12a42d710c1063 (127.31.15.193:41235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 582a7757f9114d2585bcec3b676f975b: Network error: Client connection negotiation failed: client connection to 127.31.15.193:41235: connect: Connection refused (error 111)
I20260809 16:37:30.721290 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 24772
I20260809 16:37:30.732019 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 24195
I20260809 16:37:50.739634 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--webserver_interface=127.31.15.254
--webserver_port=39119
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:32839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:37:50.820253 25044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:50.820446 25044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:50.820466 25044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:50.821575 25044 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:37:50.821619 25044 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:37:50.821630 25044 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:37:50.821652 25044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:37:50.821678 25044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:50.821702 25044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:50.821722 25044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:37:50.821746 25044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:37:50.823218 25044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:32839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=39119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:50.823468 25044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:50.823701 25044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:37:50.832536 25044 server_base.cc:1034] running on GCE node
W20260809 16:37:50.832508 25054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:50.832470 25053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:50.832485 25057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:50.832932 25044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:50.833228 25044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:50.834370 25044 hybrid_clock.cc:648] HybridClock initialized: now 1786293470834359 us; error 27 us; skew 500 ppm
Aug 09 16:37:50 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:50.836938 25044 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:37:50.837388 25044 webserver.cc:460] Webserver started at http://127.31.15.254:39119/ using document root <none> and password file <none>
I20260809 16:37:50.837615 25044 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:50.837680 25044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:50.838964 25044 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:50.839550 25068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:50.839702 25044 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:50.839787 25044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "1f480fd241534f1f81686e6ff24d7321"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "6dc56cdcaa74c8df21cd8fd84e4a60d6"
server_key_iv: "669d9a1c906992f582fa673244528fbf"
server_key_version: "encryptionkey@0"
I20260809 16:37:50.840243 25044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:50.863739 25044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:50.865756 25044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:50.865895 25044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:50.869837 25044 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:32839
I20260809 16:37:50.869875 25143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:32839 every 8 connection(s)
I20260809 16:37:50.870253 25044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:37:50.871762 25144 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:37:50.872373 25144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap starting.
I20260809 16:37:50.875420 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 25044
I20260809 16:37:50.875736 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:41235
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=44315
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:50.881834 25144 log.cc:824] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:50.883728 25144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:50.884018 25144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap complete.
I20260809 16:37:50.886170 25144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:50.886440 25144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Initialized, Role: FOLLOWER
I20260809 16:37:50.886583 25144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:50.886646 25144 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:37:50.886688 25144 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:37:50.886735 25144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 1 FOLLOWER]: Advancing to term 2
I20260809 16:37:50.887766 25144 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:50.887856 25144 leader_election.cc:302] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f480fd241534f1f81686e6ff24d7321; no voters:
I20260809 16:37:50.888016 25144 leader_election.cc:288] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 16:37:50.888053 25152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 16:37:50.888213 25152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 LEADER]: Becoming Leader. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Running, Role: LEADER
I20260809 16:37:50.888286 25144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:37:50.888324 25152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:37:50.888630 25152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f480fd241534f1f81686e6ff24d7321. Latest consensus state: current_term: 2 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:37:50.888867 25152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:50.888799 25153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:37:50.889258 25153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:37:50.889358 25171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:37:50.890143 25171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:37:50.890286 25171 catalog_manager.cc:1259] Loaded cluster ID: 5bddf68743fc428bbbd3cced235a78e1
I20260809 16:37:50.890564 25171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:37:50.891763 25171 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:37:50.891932 25171 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Loaded TSK: 0
I20260809 16:37:50.901561 25171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Generated new TSK 1
I20260809 16:37:50.901647 25171 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:37:50.960842 25148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:50.961036 25148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:50.961058 25148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:50.962153 25148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:50.962193 25148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:50.962217 25148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:37:50.963624 25148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:41235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=44315
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:50.963835 25148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:50.964047 25148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:50.969722 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:50.969733 25194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:50.969952 25148 server_base.cc:1034] running on GCE node
W20260809 16:37:50.970027 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:50.970247 25148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:50.970526 25148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:50.971704 25148 hybrid_clock.cc:648] HybridClock initialized: now 1786293470971687 us; error 40 us; skew 500 ppm
Aug 09 16:37:50 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:50.974633 25148 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:37:50.975154 25148 webserver.cc:460] Webserver started at http://127.31.15.193:44315/ using document root <none> and password file <none>
I20260809 16:37:50.975442 25148 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:50.975533 25148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:50.977154 25148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:37:50.977946 25210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:50.978135 25148 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:50.978230 25148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "eb320913384c46359f12a42d710c1063"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "26234043cc2d7b21ccd963ea2817b471"
server_key_iv: "3115063ec45929dd64fd418137256fb7"
server_key_version: "encryptionkey@0"
I20260809 16:37:50.978562 25148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:51.002369 25148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:51.004688 25148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:51.004815 25148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:51.005069 25148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:51.005520 25217 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:37:51.007436 25148 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:37:51.007481 25148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 16:37:51.007516 25148 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:37:51.008297 25148 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:37:51.008376 25148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:51.008422 25217 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Bootstrap starting.
I20260809 16:37:51.015517 25148 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:41235
I20260809 16:37:51.015591 25404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:41235 every 8 connection(s)
I20260809 16:37:51.015897 25148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:37:51.018451 25217 log.cc:824] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:51.020581 25217 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:37:51.020828 25217 tablet_bootstrap.cc:492] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Bootstrap complete.
I20260809 16:37:51.021675 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 25148
I20260809 16:37:51.021777 25217 ts_tablet_manager.cc:1397] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260809 16:37:51.021956 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:33001
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=39961
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:51.022778 25217 raft_consensus.cc:348] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:51.023044 25217 raft_consensus.cc:729] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb320913384c46359f12a42d710c1063, State: Initialized, Role: FOLLOWER
I20260809 16:37:51.023204 25217 consensus_queue.cc:260] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } } peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } }
I20260809 16:37:51.023509 25217 ts_tablet_manager.cc:1428] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260809 16:37:51.026407 25408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.016530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:56517)
0809 16:37:51.016705 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.016747 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.017227 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.017629 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.017633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.017893 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:51.018074 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.018080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.019385 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.019386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.020199 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.020202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.020794 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.020795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.020930 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.021429 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:51.021446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:51.023154 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:51.024915 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.024922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.024925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.025140 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.025360 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.025361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.025362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.025465 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.025806 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:51.025813 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:51.025941 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.026027 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.026206 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260809 16:37:51.026402 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.016647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:56517 (local address 127.31.15.254:32839)
0809 16:37:51.016841 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.016850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.016893 (+ 43us) server_negotiation.cc:407] Connection header received
0809 16:37:51.017428 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.017433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.017476 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.017587 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.018186 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.019253 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.020280 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.020700 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.020798 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.023264 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.023288 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.023288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.023300 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.024836 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.025213 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.025216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.025217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.025302 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.025521 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.025522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.025522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.025742 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.025814 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.026091 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.026213 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260809 16:37:51.027024 25405 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:51.027132 25405 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:51.027341 25405 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:51.028079 25098 ts_manager.cc:194] Registered new tserver with Master: eb320913384c46359f12a42d710c1063 (127.31.15.193:41235)
I20260809 16:37:51.029067 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:56517
I20260809 16:37:51.030040 25405 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293470, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.193@KRBTEST.COM
I20260809 16:37:51.036262 25414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.028724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60324 (local address 127.31.15.193:41235)
0809 16:37:51.028866 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.028871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.029464 (+ 593us) server_negotiation.cc:407] Connection header received
0809 16:37:51.029558 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.029560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.029574 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.029627 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.030008 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.030822 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.031578 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.031857 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.031911 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.033805 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.033812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.033812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.033818 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.035131 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.035557 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.035559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.035559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.035612 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.035784 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.035784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.035785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.035901 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.035955 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.036115 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.036201 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 16:37:51.036324 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.028645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.0.0.1:60324)
0809 16:37:51.029355 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.029389 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.029458 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.029643 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.029649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.029894 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:51.029936 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.029937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.030889 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.030890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.031501 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.031503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.031919 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.031921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.031990 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.032392 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:51.032396 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:51.033722 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:51.035201 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.035208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.035210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.035482 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.035668 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.035668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.035669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.035733 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.035948 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:51.035953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:51.036004 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.036085 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.036222 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":82,"threads_started":1}
I20260809 16:37:51.036685 25272 tablet_service.cc:1514] Processing DeleteTablet for tablet 582a7757f9114d2585bcec3b676f975b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:30 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60324
I20260809 16:37:51.037150 25417 tablet_replica.cc:330] stopping tablet replica
I20260809 16:37:51.037240 25417 raft_consensus.cc:2229] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:51.037304 25417 raft_consensus.cc:2258] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:51.037583 25417 ts_tablet_manager.cc:1905] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:51.039615 25417 ts_tablet_manager.cc:1918] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:51.039683 25417 log.cc:1196] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal/wals/582a7757f9114d2585bcec3b676f975b
I20260809 16:37:51.039868 25417 ts_tablet_manager.cc:1939] T 582a7757f9114d2585bcec3b676f975b P eb320913384c46359f12a42d710c1063: Deleting consensus metadata
I20260809 16:37:51.040356 25073 catalog_manager.cc:4841] TS eb320913384c46359f12a42d710c1063 (127.31.15.193:41235): tablet 582a7757f9114d2585bcec3b676f975b (table test-table [id=0e22eea875f344c0a88719e073711d27]) successfully deleted
W20260809 16:37:51.106467 25412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:51.106665 25412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:51.106688 25412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:51.107888 25412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:51.107939 25412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:51.107975 25412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:37:51.109462 25412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:33001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=39961
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:51.109655 25412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:51.109833 25412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:51.114478 25428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:51.114526 25425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:51.114535 25423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:51.114809 25412 server_base.cc:1034] running on GCE node
I20260809 16:37:51.115011 25412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:51.115233 25412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:51.116389 25412 hybrid_clock.cc:648] HybridClock initialized: now 1786293471116370 us; error 37 us; skew 500 ppm
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:51.118765 25412 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:37:51.119138 25412 webserver.cc:460] Webserver started at http://127.31.15.194:39961/ using document root <none> and password file <none>
I20260809 16:37:51.119354 25412 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:51.119413 25412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:51.120640 25412 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:51.121135 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.121279 25412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.121358 25412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "15f341b4f0584a16a90d5c506ed9611e"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "0142b50f210c121e17f2d6a91643ea9e"
server_key_iv: "745496b8b2ff6d6f163ce40c2ad24775"
server_key_version: "encryptionkey@0"
I20260809 16:37:51.121687 25412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:51.151351 25412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:51.153049 25412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:51.153160 25412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:51.153335 25412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:51.153604 25412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:51.153635 25412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.153656 25412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:51.153672 25412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.159055 25412 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33001
I20260809 16:37:51.159099 25624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33001 every 8 connection(s)
I20260809 16:37:51.159428 25412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:51.167706 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.160168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:51139 (local address 127.31.15.254:32839)
0809 16:37:51.160233 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.160235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.160416 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:37:51.160879 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.160880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.160891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.160940 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.161807 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.162291 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.163288 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.163532 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.163587 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.165576 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.165577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.165578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.165584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.166648 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.166971 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.166972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.166972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.167015 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.167184 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.167185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.167185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.167305 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.167359 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.167568 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.167639 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:37:51.167829 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.160142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:51139)
0809 16:37:51.160302 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.160338 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.160799 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.161013 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.161017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.161554 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:51.161740 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.161745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.162351 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.162352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.163193 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.163196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.163613 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.163614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.163739 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.164137 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:51.164145 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:51.165498 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:51.166711 (+ 1213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.166716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.166718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.166918 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.167058 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.167058 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.167059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.167144 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.167354 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:51.167357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:51.167463 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.167548 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.167661 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260809 16:37:51.167922 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 25412
I20260809 16:37:51.168179 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:42517
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=35751
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:37:51.168638 25626 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:51.168725 25626 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:51.168888 25626 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:51.169281 25098 ts_manager.cc:194] Registered new tserver with Master: 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001)
I20260809 16:37:51.169634 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:51139
W20260809 16:37:51.248628 25629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:37:51.248822 25629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:37:51.248843 25629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:37:51.249922 25629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:37:51.249958 25629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:37:51.249981 25629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:37:51.251369 25629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:42517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=35751
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:37:51.251547 25629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:37:51.251729 25629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:37:51.256229 25644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:51.256390 25629 server_base.cc:1034] running on GCE node
W20260809 16:37:51.256263 25642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:37:51.256230 25640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:37:51.256675 25629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:37:51.256906 25629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:37:51.258041 25629 hybrid_clock.cc:648] HybridClock initialized: now 1786293471258025 us; error 33 us; skew 500 ppm
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:37:51.260457 25629 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:37:51.260813 25629 webserver.cc:460] Webserver started at http://127.31.15.195:35751/ using document root <none> and password file <none>
I20260809 16:37:51.260977 25629 fs_manager.cc:359] Metadata directory not provided
I20260809 16:37:51.261026 25629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:37:51.262221 25629 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:37:51.262763 25652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.262940 25629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.263022 25629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "d6857373d6f0366923838d515dc818e2"
server_key_iv: "d4651c516cf78ae216dcf14d7d35e87b"
server_key_version: "encryptionkey@0"
I20260809 16:37:51.263350 25629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:37:51.287664 25629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:37:51.289856 25629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:37:51.289989 25629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:37:51.290192 25629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:37:51.290475 25629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:37:51.290510 25629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.290537 25629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:37:51.290562 25629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:37:51.296016 25629 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:42517
I20260809 16:37:51.296083 25852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:42517 every 8 connection(s)
I20260809 16:37:51.296396 25629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:37:51 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293471, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:37:51.303789 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 25629
I20260809 16:37:51.305078 25409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.297102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:40657 (local address 127.31.15.254:32839)
0809 16:37:51.297158 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.297160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.297355 (+ 195us) server_negotiation.cc:407] Connection header received
0809 16:37:51.297859 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.297861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.297873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.297939 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.298882 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.299413 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.300378 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.300572 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.300628 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.302547 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.302548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.302549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.302555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.303657 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.304036 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.304037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.304038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.304092 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.304459 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.304461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.304461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.304605 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.304677 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.304908 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.304992 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:37:51.305143 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.297068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:40657)
0809 16:37:51.297236 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.297271 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.297756 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.298019 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.298023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.298593 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:37:51.298779 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.298783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.299492 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.299493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.300295 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.300297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.300641 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.300642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.300753 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.301151 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:37:51.301159 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:37:51.302459 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:37:51.303726 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.303731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.303733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.303959 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.304252 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:37:51.304253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:37:51.304254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:37:51.304383 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:37:51.304665 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:37:51.304670 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:37:51.304780 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.304859 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.304975 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 16:37:51.305600 25853 heartbeater.cc:347] Connected to a master server at 127.31.15.254:32839
I20260809 16:37:51.305689 25853 heartbeater.cc:464] Registering TS with master...
I20260809 16:37:51.305863 25853 heartbeater.cc:511] Master 127.31.15.254:32839 requested a full tablet report, sending...
I20260809 16:37:51.306272 25098 ts_manager.cc:194] Registered new tserver with Master: 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
I20260809 16:37:51.306656 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:40657
I20260809 16:37:51.309916 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.304159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.31.15.254:32839)
0809 16:37:51.304266 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.304268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.304345 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:37:51.304386 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.304386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.304398 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.304452 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.305126 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.305797 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.306701 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.306942 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.306994 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.308092 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.308093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.308093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.308099 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.309154 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.309425 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.309425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.309425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.309469 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.309577 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.309577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.309578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.309655 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.309765 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.309789 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.309848 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":57,"threads_started":1}
I20260809 16:37:51.310571 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:37:51.313092 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.310860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41714 (local address 127.31.15.254:32839)
0809 16:37:51.310917 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.310919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.310980 (+ 61us) server_negotiation.cc:407] Connection header received
0809 16:37:51.311048 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.311049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.311085 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.311142 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:51.311237 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.311707 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.312378 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.312575 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.312625 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.312762 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:51.312806 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 16:37:51.313052 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41714: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 16:37:51.313100 24978 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.31.15.254:32839)
W20260809 16:37:51.313136 25859 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41714: token expired
I20260809 16:37:51.317888 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.313287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41728 (local address 127.31.15.254:32839)
0809 16:37:51.313329 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.313331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.313369 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:37:51.313409 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.313410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.313417 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.313463 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:37:51.313824 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.314235 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.314822 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.314986 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.315025 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.315999 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:37:51.316000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:37:51.316001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:37:51.316007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:37:51.316993 (+ 986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.317272 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.317273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.317273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.317319 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:37:51.317477 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:37:51.317478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:37:51.317478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:37:51.317569 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:37:51.317648 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.317744 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.317811 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:37:51.318821 25098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:37:51.320395 25098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:37:51.322345 25272 tablet_service.cc:1467] Processing CreateTablet for tablet dc7432a8e81643168af586426cd5d50b (DEFAULT_TABLE table=test-table [id=8c59ca23b222407ea4dcf30a33bbe4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:51.322528 25272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7432a8e81643168af586426cd5d50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:51.323920 25217 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Bootstrap starting.
I20260809 16:37:51.324268 25217 tablet_bootstrap.cc:654] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:51.324842 25217 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: No bootstrap required, opened a new log
I20260809 16:37:51.324894 25217 ts_tablet_manager.cc:1397] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:51.325237 25217 raft_consensus.cc:348] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.325315 25217 raft_consensus.cc:374] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:51.325362 25217 raft_consensus.cc:729] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb320913384c46359f12a42d710c1063, State: Initialized, Role: FOLLOWER
I20260809 16:37:51.325409 25217 consensus_queue.cc:260] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.325516 25217 ts_tablet_manager.cc:1428] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 16:37:51.326094 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.322068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:42517 (local address 127.0.0.1:51466)
0809 16:37:51.322241 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.322251 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.322296 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.322685 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.322687 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.322689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:51.322731 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.322732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.324034 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.324035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.325053 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.325056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.325900 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.325901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.325948 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.325964 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.326006 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.326034 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":5,"thread_start_us":92,"threads_started":1}
I20260809 16:37:51.326153 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.322333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.31.15.195:42517)
0809 16:37:51.322486 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.322490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.322501 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:37:51.322536 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.322538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.322553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.322662 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:51.322803 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.323872 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.325142 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.325829 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.325900 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.325955 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.326032 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.326072 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 16:37:51.326315 25413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.322020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33001 (local address 127.0.0.1:35242)
0809 16:37:51.322053 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.322063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.322099 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.322686 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.322687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.322689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:51.322740 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.322741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.324190 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.324191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.325215 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.325218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.326102 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.326102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.326160 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.326164 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.326217 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.326259 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":8}
I20260809 16:37:51.326576 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.322151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35242 (local address 127.31.15.194:33001)
0809 16:37:51.322517 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.322520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.322529 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:37:51.322562 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.322564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.322580 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.322844 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:51.322946 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.324120 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.325326 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.326025 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.326085 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.326137 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.326454 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.326501 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":88,"threads_started":1}
I20260809 16:37:51.326978 25739 tablet_service.cc:1467] Processing CreateTablet for tablet dc7432a8e81643168af586426cd5d50b (DEFAULT_TABLE table=test-table [id=8c59ca23b222407ea4dcf30a33bbe4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:51.327117 25511 tablet_service.cc:1467] Processing CreateTablet for tablet dc7432a8e81643168af586426cd5d50b (DEFAULT_TABLE table=test-table [id=8c59ca23b222407ea4dcf30a33bbe4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:37:51.327327 25739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7432a8e81643168af586426cd5d50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:51.327443 25511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc7432a8e81643168af586426cd5d50b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:37:51.330010 25865 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Bootstrap starting.
I20260809 16:37:51.330109 25866 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Bootstrap starting.
I20260809 16:37:51.330590 25865 tablet_bootstrap.cc:654] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:51.330613 25866 tablet_bootstrap.cc:654] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Neither blocks nor log segments found. Creating new log.
I20260809 16:37:51.331054 25866 log.cc:824] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:51.331053 25865 log.cc:824] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:37:51.331749 25865 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: No bootstrap required, opened a new log
I20260809 16:37:51.331810 25865 ts_tablet_manager.cc:1397] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:37:51.333060 25865 raft_consensus.cc:348] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.333189 25865 raft_consensus.cc:374] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:51.333220 25865 raft_consensus.cc:729] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a12282ec58a4e9eb724fa32f2ee2321, State: Initialized, Role: FOLLOWER
I20260809 16:37:51.333333 25865 consensus_queue.cc:260] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.333560 25865 ts_tablet_manager.cc:1428] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:51.333611 25853 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
I20260809 16:37:51.333936 25866 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: No bootstrap required, opened a new log
I20260809 16:37:51.334009 25866 ts_tablet_manager.cc:1397] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 16:37:51.335522 25866 raft_consensus.cc:348] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.335654 25866 raft_consensus.cc:374] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:37:51.335675 25866 raft_consensus.cc:729] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f341b4f0584a16a90d5c506ed9611e, State: Initialized, Role: FOLLOWER
I20260809 16:37:51.335772 25866 consensus_queue.cc:260] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.335990 25866 ts_tablet_manager.cc:1428] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:37:51.336041 25626 heartbeater.cc:503] Master 127.31.15.254:32839 was elected leader, sending a full tablet report...
W20260809 16:37:51.410012 25627 tablet.cc:2369] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:51.516732 25407 tablet.cc:2369] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 16:37:51.546980 25855 tablet.cc:2369] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:37:51.621559 25874 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:37:51.621699 25874 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.622102 25874 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517), eb320913384c46359f12a42d710c1063 (127.31.15.193:41235)
I20260809 16:37:51.625000 25863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.622397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:40635 (local address 127.31.15.195:42517)
0809 16:37:51.622479 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.622482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.622559 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:37:51.622597 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.622598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.622608 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.622655 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:51.622800 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.623264 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624283 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.624706 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624765 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.624776 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.624898 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.624935 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 16:37:51.625046 25875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.622335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:42517 (local address 127.31.15.194:40635)
0809 16:37:51.622495 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.622504 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.622552 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.622665 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.622666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.622667 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:51.622738 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.622740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.623355 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.623356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.624197 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.624200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.624783 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.624840 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.624843 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.624885 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.624922 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":69,"thread_start_us":62,"threads_started":1}
I20260809 16:37:51.625072 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.622429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:53203 (local address 127.31.15.193:41235)
0809 16:37:51.622566 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.622567 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.622574 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:37:51.622599 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.622600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.622624 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.622709 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:37:51.622864 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.623378 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624296 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.624758 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624822 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.624934 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.624953 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.625003 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260809 16:37:51.625140 25628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.622333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.31.15.194:53203)
0809 16:37:51.622392 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.622404 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:37:51.622481 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:37:51.622730 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:37:51.622731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:37:51.622732 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:37:51.622794 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.622796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.623444 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.623445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.624215 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:37:51.624217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.624826 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.624827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:37:51.624898 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.624901 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:37:51.624942 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:37:51.624978 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:37:51.625408 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7432a8e81643168af586426cd5d50b" candidate_uuid: "15f341b4f0584a16a90d5c506ed9611e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" is_pre_election: true
I20260809 16:37:51.625418 25313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7432a8e81643168af586426cd5d50b" candidate_uuid: "15f341b4f0584a16a90d5c506ed9611e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb320913384c46359f12a42d710c1063" is_pre_election: true
I20260809 16:37:51.625560 25778 raft_consensus.cc:2454] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f341b4f0584a16a90d5c506ed9611e in term 0.
I20260809 16:37:51.625568 25313 raft_consensus.cc:2454] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15f341b4f0584a16a90d5c506ed9611e in term 0.
I20260809 16:37:51.625783 25439 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321; no voters:
I20260809 16:37:51.625962 25874 raft_consensus.cc:2781] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:37:51.626003 25874 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:37:51.626025 25874 raft_consensus.cc:3037] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:51.626705 25874 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.626827 25874 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [CANDIDATE]: Term 1 election: Requested vote from peers 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517), eb320913384c46359f12a42d710c1063 (127.31.15.193:41235)
I20260809 16:37:51.626981 25778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7432a8e81643168af586426cd5d50b" candidate_uuid: "15f341b4f0584a16a90d5c506ed9611e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
I20260809 16:37:51.627004 25313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc7432a8e81643168af586426cd5d50b" candidate_uuid: "15f341b4f0584a16a90d5c506ed9611e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb320913384c46359f12a42d710c1063"
I20260809 16:37:51.627068 25313 raft_consensus.cc:3037] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:51.627065 25778 raft_consensus.cc:3037] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:37:51.627903 25778 raft_consensus.cc:2454] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f341b4f0584a16a90d5c506ed9611e in term 1.
I20260809 16:37:51.627903 25313 raft_consensus.cc:2454] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15f341b4f0584a16a90d5c506ed9611e in term 1.
I20260809 16:37:51.628073 25439 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321; no voters:
I20260809 16:37:51.628152 25874 raft_consensus.cc:2781] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:37:51.628264 25874 raft_consensus.cc:686] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 1 LEADER]: Becoming Leader. State: Replica: 15f341b4f0584a16a90d5c506ed9611e, State: Running, Role: LEADER
I20260809 16:37:51.628342 25874 consensus_queue.cc:237] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:37:51.629138 25098 catalog_manager.cc:5495] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e reported cstate change: term changed from 0 to 1, leader changed from <none> to 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194). New cstate: current_term: 1 leader_uuid: "15f341b4f0584a16a90d5c506ed9611e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } health_report { overall_health: UNKNOWN } } }
I20260809 16:37:51.677727 25862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.675563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.31.15.194:33001)
0809 16:37:51.675619 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:37:51.675621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:37:51.675629 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:37:51.675666 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:37:51.675666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:37:51.675683 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:37:51.675742 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:37:51.675854 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.676318 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.677020 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:37:51.677204 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:37:51.677258 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:37:51.677407 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:37:51.677507 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:37:51.677576 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:37:51.677614 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:37:51.677634 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:37:51.679400 25778 raft_consensus.cc:1261] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Refusing update from remote peer 15f341b4f0584a16a90d5c506ed9611e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:51.679409 25313 raft_consensus.cc:1261] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Refusing update from remote peer 15f341b4f0584a16a90d5c506ed9611e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:37:51.679714 25874 consensus_queue.cc:1035] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:51.679857 25878 consensus_queue.cc:1035] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:37:51.682022 25881 mvcc.cc:205] Tried to move back new op lower bound from 7316658059996229632 to 7316658059790180352. Current Snapshot: MvccSnapshot[applied={T|T < 7316658059996229632}]
I20260809 16:37:51.682793 25883 mvcc.cc:205] Tried to move back new op lower bound from 7316658059996229632 to 7316658059790180352. Current Snapshot: MvccSnapshot[applied={T|T < 7316658059996229632}]
I20260809 16:37:51.684185 25097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:51.684285 25097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:37:51.685000 25097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Sending DeleteTablet for 3 replicas of tablet dc7432a8e81643168af586426cd5d50b
I20260809 16:37:51.685278 25511 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7432a8e81643168af586426cd5d50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:51 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:35242
I20260809 16:37:51.685310 25272 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7432a8e81643168af586426cd5d50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:51 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:60324
I20260809 16:37:51.685290 25739 tablet_service.cc:1514] Processing DeleteTablet for tablet dc7432a8e81643168af586426cd5d50b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:37:51 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:51466
I20260809 16:37:51.685531 25897 tablet_replica.cc:330] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: stopping tablet replica
I20260809 16:37:51.685557 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 25148
I20260809 16:37:51.700592 25897 raft_consensus.cc:2229] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:37:51.700768 25897 raft_consensus.cc:2258] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:51.703756 25897 ts_tablet_manager.cc:1905] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:51.705353 25897 ts_tablet_manager.cc:1918] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:51.705428 25897 log.cc:1196] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/dc7432a8e81643168af586426cd5d50b
I20260809 16:37:51.705647 25897 ts_tablet_manager.cc:1939] T dc7432a8e81643168af586426cd5d50b P 15f341b4f0584a16a90d5c506ed9611e: Deleting consensus metadata
I20260809 16:37:51.706107 25073 catalog_manager.cc:4841] TS 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): tablet dc7432a8e81643168af586426cd5d50b (table test-table [id=8c59ca23b222407ea4dcf30a33bbe4fd]) successfully deleted
W20260809 16:37:51.713685 25073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:41235 (error 108)
I20260809 16:37:51.713826 25899 tablet_replica.cc:330] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: stopping tablet replica
I20260809 16:37:51.713963 25899 raft_consensus.cc:2229] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:37:51.714110 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 25412
I20260809 16:37:51.715641 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:37:51.714203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:41235 (local address 127.0.0.1:60338)
0809 16:37:51.715558 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0809 16:37:51.715605 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:41235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1325}
W20260809 16:37:51.717376 25073 catalog_manager.cc:4584] TS eb320913384c46359f12a42d710c1063 (127.31.15.193:41235): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc7432a8e81643168af586426cd5d50b: Network error: Client connection negotiation failed: client connection to 127.31.15.193:41235: connect: Connection refused (error 111)
I20260809 16:37:51.726506 25899 raft_consensus.cc:2258] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:37:51.727012 25899 ts_tablet_manager.cc:1905] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:37:51.728493 25899 ts_tablet_manager.cc:1918] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:37:51.728569 25899 log.cc:1196] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/dc7432a8e81643168af586426cd5d50b
I20260809 16:37:51.728796 25899 ts_tablet_manager.cc:1939] T dc7432a8e81643168af586426cd5d50b P 7a12282ec58a4e9eb724fa32f2ee2321: Deleting consensus metadata
I20260809 16:37:51.729209 25071 catalog_manager.cc:4841] TS 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): tablet dc7432a8e81643168af586426cd5d50b (table test-table [id=8c59ca23b222407ea4dcf30a33bbe4fd]) successfully deleted
I20260809 16:37:51.730449 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 25629
I20260809 16:37:51.740224 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 25044
I20260809 16:38:11.751454 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--webserver_interface=127.31.15.254
--webserver_port=39119
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:32839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:11.832793 25901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:11.832990 25901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:11.833019 25901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:11.834148 25901 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 16:38:11.834188 25901 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 16:38:11.834204 25901 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 16:38:11.834218 25901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:11.834234 25901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:11.834252 25901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:11.834273 25901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:11.834296 25901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:11.835772 25901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:32839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=39119
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:11.836002 25901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:11.836200 25901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:38:11.840939 25901 server_base.cc:1034] running on GCE node
W20260809 16:38:11.840996 25917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:11.840968 25916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:11.841111 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:11.841434 25901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:11.841679 25901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:11.842832 25901 hybrid_clock.cc:648] HybridClock initialized: now 1786293491842820 us; error 29 us; skew 500 ppm
Aug 09 16:38:11 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:11.845295 25901 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:11.845733 25901 webserver.cc:460] Webserver started at http://127.31.15.254:39119/ using document root <none> and password file <none>
I20260809 16:38:11.845953 25901 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:11.845999 25901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:11.847211 25901 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:38:11.847880 25933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:11.848037 25901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:11.848110 25901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "1f480fd241534f1f81686e6ff24d7321"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "6dc56cdcaa74c8df21cd8fd84e4a60d6"
server_key_iv: "669d9a1c906992f582fa673244528fbf"
server_key_version: "encryptionkey@0"
I20260809 16:38:11.848412 25901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:11.875078 25901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:11.877259 25901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:11.877403 25901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:11.881431 25901 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:32839
I20260809 16:38:11.881484 26029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:32839 every 8 connection(s)
I20260809 16:38:11.881846 25901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:11.883339 26030 sys_catalog.cc:263] Verifying existing consensus state
I20260809 16:38:11.884023 26030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap starting.
I20260809 16:38:11.887348 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 25901
I20260809 16:38:11.887687 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:41235
--local_ip_for_outbound_sockets=127.31.15.193
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=44315
--webserver_interface=127.31.15.193
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:38:11.893843 26030 log.cc:824] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:11.896055 26030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:38:11.896345 26030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Bootstrap complete.
I20260809 16:38:11.898624 26030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:38:11.898847 26030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Initialized, Role: FOLLOWER
I20260809 16:38:11.899001 26030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:38:11.899065 26030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:11.899106 26030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:11.899151 26030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 2 FOLLOWER]: Advancing to term 3
I20260809 16:38:11.900159 26030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:38:11.900249 26030 leader_election.cc:302] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f480fd241534f1f81686e6ff24d7321; no voters:
I20260809 16:38:11.900417 26030 leader_election.cc:288] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 16:38:11.900475 26035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 16:38:11.900599 26035 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [term 3 LEADER]: Becoming Leader. State: Replica: 1f480fd241534f1f81686e6ff24d7321, State: Running, Role: LEADER
I20260809 16:38:11.900683 26030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:11.900753 26035 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } }
I20260809 16:38:11.901064 26036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:38:11.901093 26035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f480fd241534f1f81686e6ff24d7321. Latest consensus state: current_term: 3 leader_uuid: "1f480fd241534f1f81686e6ff24d7321" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f480fd241534f1f81686e6ff24d7321" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 32839 } } }
I20260809 16:38:11.901149 26036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:11.901162 26035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:11.901343 26043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:11.902269 26043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:11.902423 26043 catalog_manager.cc:1259] Loaded cluster ID: 5bddf68743fc428bbbd3cced235a78e1
I20260809 16:38:11.902621 26043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:11.903915 26043 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:11.904102 26043 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Loaded TSK: 0
I20260809 16:38:11.904145 26043 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1f480fd241534f1f81686e6ff24d7321: Loaded TSK: 1
F20260809 16:38:11.904371 26043 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f56e7f939ad google::LogMessage::Fail() at ??:0
@ 0x7f56e7f9688a google::LogMessage::SendToLog() at ??:0
@ 0x7f56e7f934de google::LogMessage::Flush() at ??:0
@ 0x7f56e7f94a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f56ea700104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f56ea6fdd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f56efeed1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f56efeed48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f56efeed594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f56efeb3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f56efebdad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f56e9e610d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f56e9e56954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f56eacba6db start_thread at ??:0
@ 0x7f56e6a9161f clone at ??:0
W20260809 16:38:11.984117 26032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:11.984310 26032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:11.984342 26032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:11.985563 26032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:11.985606 26032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:11.985641 26032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:11.987036 26032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:41235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=44315
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:11.987294 26032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:11.987505 26032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:38:11.992247 26032 server_base.cc:1034] running on GCE node
W20260809 16:38:11.992264 26101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:11.992338 26102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:11.992233 26104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:11.992626 26032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:11.992841 26032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:11.993973 26032 hybrid_clock.cc:648] HybridClock initialized: now 1786293491993948 us; error 40 us; skew 500 ppm
Aug 09 16:38:11 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:11.996480 26032 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:38:11.996876 26032 webserver.cc:460] Webserver started at http://127.31.15.193:44315/ using document root <none> and password file <none>
I20260809 16:38:11.997076 26032 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:11.997123 26032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:11.998441 26032 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:11.999112 26116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:11.999261 26032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:11.999373 26032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "eb320913384c46359f12a42d710c1063"
format_stamp: "Formatted at 2026-08-09 16:37:29 on dist-test-slave-21b9"
server_key: "26234043cc2d7b21ccd963ea2817b471"
server_key_iv: "3115063ec45929dd64fd418137256fb7"
server_key_version: "encryptionkey@0"
I20260809 16:38:11.999696 26032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:12.006669 26032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:12.008930 26032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:12.009086 26032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:12.009341 26032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:12.009829 26124 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 16:38:12.012362 26032 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 16:38:12.012444 26032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260809 16:38:12.012542 26032 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 16:38:12.013412 26032 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 16:38:12.013455 26032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 16:38:12.013577 26124 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Bootstrap starting.
I20260809 16:38:12.021088 26032 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:41235
I20260809 16:38:12.021222 26301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:41235 every 8 connection(s)
I20260809 16:38:12.021597 26032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:38:12.023550 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 26032
I20260809 16:38:12.023876 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:33001
--local_ip_for_outbound_sockets=127.31.15.194
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=39961
--webserver_interface=127.31.15.194
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:38:12.024930 26124 log.cc:824] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:12.027088 26124 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 16:38:12.027405 26124 tablet_bootstrap.cc:492] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Bootstrap complete.
I20260809 16:38:12.028237 26124 ts_tablet_manager.cc:1397] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260809 16:38:12.029263 26124 raft_consensus.cc:348] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:12.029491 26124 raft_consensus.cc:729] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb320913384c46359f12a42d710c1063, State: Initialized, Role: FOLLOWER
I20260809 16:38:12.029627 26124 consensus_queue.cc:260] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:12.029865 26124 ts_tablet_manager.cc:1428] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260809 16:38:12.101833 26304 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:38:12.022403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:53919)
0809 16:38:12.022635 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.022685 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:12.023293 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:12.101629 (+ 78336us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
W20260809 16:38:12.102080 26302 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:32839 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:32839: Client connection negotiation failed: client connection to 127.31.15.254:32839: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 16:38:12.102401 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.102294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:42611)
0809 16:38:12.102341 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.102381 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:38:12.102731 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.102654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:56025)
0809 16:38:12.102694 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.102716 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 16:38:12.102841 26302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 16:38:12.112367 26308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:12.112589 26308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:12.112613 26308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:12.113705 26308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:12.113739 26308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:12.113762 26308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:12.115128 26308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:33001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=39961
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:12.115348 26308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:12.115564 26308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:12.120208 26328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:12.120261 26308 server_base.cc:1034] running on GCE node
W20260809 16:38:12.120190 26322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:12.120185 26319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:12.120545 26308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:12.120755 26308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:12.121886 26308 hybrid_clock.cc:648] HybridClock initialized: now 1786293492121875 us; error 29 us; skew 500 ppm
Aug 09 16:38:12 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293492, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:12.124331 26308 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:38:12.124724 26308 webserver.cc:460] Webserver started at http://127.31.15.194:39961/ using document root <none> and password file <none>
I20260809 16:38:12.124934 26308 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:12.124984 26308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:12.126201 26308 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:12.126686 26339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.126837 26308 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.126917 26308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "15f341b4f0584a16a90d5c506ed9611e"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "0142b50f210c121e17f2d6a91643ea9e"
server_key_iv: "745496b8b2ff6d6f163ce40c2ad24775"
server_key_version: "encryptionkey@0"
I20260809 16:38:12.127221 26308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:12.138437 26308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:12.140137 26308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:12.140265 26308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:12.140457 26308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:12.140743 26308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:12.140775 26308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.140802 26308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:12.140817 26308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.146340 26308 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33001
I20260809 16:38:12.146399 26508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33001 every 8 connection(s)
I20260809 16:38:12.146739 26308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:12.147898 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.147449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:48935)
0809 16:38:12.147633 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.147720 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
W20260809 16:38:12.148320 26510 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:32839 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:32839: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
I20260809 16:38:12.148619 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.148528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:48029)
0809 16:38:12.148569 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.148595 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 16:38:12.148923 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.148786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:60481)
0809 16:38:12.148887 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.148905 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
W20260809 16:38:12.149004 26510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:38:12.153071 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 26308
I20260809 16:38:12.153338 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:42517
--local_ip_for_outbound_sockets=127.31.15.195
--tserver_master_addrs=127.31.15.254:32839
--webserver_port=35751
--webserver_interface=127.31.15.195
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:12.235661 26516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:12.235862 26516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:12.235884 26516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:12.237035 26516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:12.237077 26516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:12.237104 26516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:12.238667 26516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:42517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=35751
--tserver_master_addrs=127.31.15.254:32839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:12.238883 26516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:12.239120 26516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:12.244125 26523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:12.244117 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:12.244225 26516 server_base.cc:1034] running on GCE node
W20260809 16:38:12.244122 26526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:12.244526 26516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:12.244750 26516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:12.245910 26516 hybrid_clock.cc:648] HybridClock initialized: now 1786293492245893 us; error 32 us; skew 500 ppm
Aug 09 16:38:12 dist-test-slave-21b9 krb5kdc[24180](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293492, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:12.248351 26516 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:38:12.248770 26516 webserver.cc:460] Webserver started at http://127.31.15.195:35751/ using document root <none> and password file <none>
I20260809 16:38:12.248967 26516 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:12.249024 26516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:12.250366 26516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:12.250993 26538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.251173 26516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:12.251257 26516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "7a12282ec58a4e9eb724fa32f2ee2321"
format_stamp: "Formatted at 2026-08-09 16:37:30 on dist-test-slave-21b9"
server_key: "d6857373d6f0366923838d515dc818e2"
server_key_iv: "d4651c516cf78ae216dcf14d7d35e87b"
server_key_version: "encryptionkey@0"
I20260809 16:38:12.251607 26516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:12.268679 26516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:12.270431 26516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:12.270540 26516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:12.270738 26516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:12.271039 26516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:12.271076 26516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.271106 26516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:12.271122 26516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:12.276620 26516 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:42517
I20260809 16:38:12.276680 26723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:42517 every 8 connection(s)
I20260809 16:38:12.276993 26516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:38:12.278100 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.277728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:49241)
0809 16:38:12.277878 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.277954 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
W20260809 16:38:12.278484 26725 heartbeater.cc:644] Failed to heartbeat to 127.31.15.254:32839 (0 consecutive failures): Network error: Failed to ping master at 127.31.15.254:32839: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
I20260809 16:38:12.278589 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 26516
I20260809 16:38:12.278761 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.278671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:43557)
0809 16:38:12.278708 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.278738 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 16:38:12.279031 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:12.278954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:53349)
0809 16:38:12.278989 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:12.279011 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:38:12.279156 26725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 16:38:13.103425 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.103195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:52149)
0809 16:38:13.103326 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:13.103359 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260809 16:38:13.149565 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.149332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:41267)
0809 16:38:13.149480 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:13.149520 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260809 16:38:13.279855 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.279588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:59437)
0809 16:38:13.279750 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:13.279784 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260809 16:38:13.809074 26734 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:13.809208 26734 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:13.809494 26734 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001), 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
Aug 09 16:38:13 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.194@KRBTEST.COM
Aug 09 16:38:13 dist-test-slave-21b9 krb5kdc[24180](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293491, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.195@KRBTEST.COM
I20260809 16:38:13.822702 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.809636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33001 (local address 127.31.15.193:35747)
0809 16:38:13.809781 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:13.809791 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:13.809859 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:13.810630 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:13.810632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:13.811002 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:13.811223 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:13.811229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.812319 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.812320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:13.813114 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:13.813115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.813662 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:13.813787 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:13.817639 (+ 3852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:13.817658 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:13.819239 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:13.821142 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:13.821151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:13.821416 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:13.821680 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:13.821681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:13.821778 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:13.822185 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:13.822193 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:13.822335 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 16:38:13.822442 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 16:38:13.822584 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":39,"thread_start_us":77,"threads_started":1}
I20260809 16:38:13.822796 26738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.809751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:35747 (local address 127.31.15.194:33001)
0809 16:38:13.809933 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:13.809946 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:13.809970 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:38:13.810396 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:13.810398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:13.810439 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:13.810578 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:13.811348 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.812217 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813197 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.813583 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813687 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:13.819369 (+ 5682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:13.819373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:13.819373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:13.819384 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:13.821060 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821493 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:13.821496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:13.821496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:13.821616 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821840 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:13.821841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:13.821841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:13.822122 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:13.822186 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:13.822546 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 16:38:13.822689 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":54,"threads_started":1}
I20260809 16:38:13.822911 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.809760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:56353 (local address 127.31.15.195:42517)
0809 16:38:13.809941 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:13.809949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:13.809969 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:38:13.810399 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:13.810400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:13.810438 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:13.810574 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:13.811352 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.812209 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813192 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.813594 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813687 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:13.819775 (+ 6088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:13.819779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:13.819780 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:13.819793 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:13.821490 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821827 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:13.821829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:13.821830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:13.821911 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:13.822092 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:13.822093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:13.822093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:13.822346 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:13.822431 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:13.822673 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 16:38:13.822799 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":50,"threads_started":1}
I20260809 16:38:13.822840 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:13.809676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:42517 (local address 127.31.15.193:56353)
0809 16:38:13.809782 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:13.809793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:13.809848 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:13.810629 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:13.810632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:13.810996 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:13.811223 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:13.811229 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.812300 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.812301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:13.813115 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:13.813117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:13.813663 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:13.813664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:13.813786 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:13.817655 (+ 3869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:13.817660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:13.819659 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:13.821562 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:13.821563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:13.821759 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:13.821968 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:13.821968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:13.821969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:13.822027 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:13.822450 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:13.822452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:13.822504 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:38:13.822602 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 16:38:13.822693 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":53,"thread_start_us":53,"threads_started":1}
W20260809 16:38:13.824055 26120 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
W20260809 16:38:13.824148 26118 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
I20260809 16:38:13.824198 26118 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb320913384c46359f12a42d710c1063; no voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321
I20260809 16:38:13.824301 26734 raft_consensus.cc:2726] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:38:14.103952 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:14.103821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:44495)
0809 16:38:14.103873 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:14.103903 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:38:14.150211 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:14.149958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:43559)
0809 16:38:14.150103 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:14.150139 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 16:38:14.280531 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:14.280299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:37007)
0809 16:38:14.280445 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:14.280490 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 16:38:15.104580 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:15.104348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:41553)
0809 16:38:15.104484 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:15.104526 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 16:38:15.150775 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:15.150589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:52197)
0809 16:38:15.150702 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:15.150736 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260809 16:38:15.281208 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:15.280984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:35723)
0809 16:38:15.281129 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:15.281165 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260809 16:38:15.471725 26750 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:15.471827 26750 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:15.471997 26750 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001), 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
W20260809 16:38:15.472458 26118 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
W20260809 16:38:15.472558 26120 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
I20260809 16:38:15.472601 26120 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb320913384c46359f12a42d710c1063; no voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321
I20260809 16:38:15.472664 26750 raft_consensus.cc:2726] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:38:16.105175 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:16.104964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:59899)
0809 16:38:16.105090 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:16.105126 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 16:38:16.151346 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:16.151125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:56849)
0809 16:38:16.151256 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:16.151307 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260809 16:38:16.281869 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:16.281640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:40939)
0809 16:38:16.281781 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:16.281824 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260809 16:38:17.105834 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:17.105613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:58575)
0809 16:38:17.105745 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:17.105782 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260809 16:38:17.151925 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:17.151698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:33459)
0809 16:38:17.151842 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:17.151877 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260809 16:38:17.160916 26759 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:17.161001 26759 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:17.161167 26759 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001), 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
W20260809 16:38:17.161502 26120 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
W20260809 16:38:17.161592 26118 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
I20260809 16:38:17.161633 26118 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb320913384c46359f12a42d710c1063; no voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321
I20260809 16:38:17.161720 26759 raft_consensus.cc:2726] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:38:17.282493 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:17.282303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:60917)
0809 16:38:17.282414 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:17.282447 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260809 16:38:18.106458 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:18.106255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:44469)
0809 16:38:18.106376 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:18.106409 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260809 16:38:18.152536 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:18.152350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:34987)
0809 16:38:18.152467 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:18.152497 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260809 16:38:18.283118 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:18.282907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:46003)
0809 16:38:18.283036 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:18.283076 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260809 16:38:19.107126 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:19.106857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:35883)
0809 16:38:19.107034 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:19.107075 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260809 16:38:19.153188 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:19.152986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:52691)
0809 16:38:19.153104 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:19.153139 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260809 16:38:19.283777 26771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:19.283558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:43281)
0809 16:38:19.283702 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:19.283736 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 16:38:20.107725 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:20.107532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:57593)
0809 16:38:20.107649 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:20.107679 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 16:38:20.153806 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:20.153604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:48195)
0809 16:38:20.153729 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:20.153765 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260809 16:38:20.284405 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:20.284183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:53753)
0809 16:38:20.284323 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:20.284361 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260809 16:38:21.108347 26778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:21.108080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:53813)
0809 16:38:21.108237 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:21.108283 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260809 16:38:21.154381 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:21.154174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:40627)
0809 16:38:21.154303 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:21.154335 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260809 16:38:21.285125 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:21.284900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:59749)
0809 16:38:21.285033 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:21.285069 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260809 16:38:21.630904 26781 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:21.631007 26781 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:21.631155 26781 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001), 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
W20260809 16:38:21.631526 26120 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
W20260809 16:38:21.631604 26118 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
I20260809 16:38:21.631666 26118 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb320913384c46359f12a42d710c1063; no voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321
I20260809 16:38:21.631752 26781 raft_consensus.cc:2726] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:38:22.109004 26783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:22.108767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:43619)
0809 16:38:22.108914 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:22.108952 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260809 16:38:22.154999 26784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:22.154781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:37311)
0809 16:38:22.154917 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:22.154948 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260809 16:38:22.285786 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:22.285558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:50871)
0809 16:38:22.285700 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:22.285741 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260809 16:38:23.109714 26787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:23.109466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:37447)
0809 16:38:23.109621 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:23.109661 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260809 16:38:23.155637 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:23.155408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:34463)
0809 16:38:23.155556 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:23.155593 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":76,"threads_started":1}
I20260809 16:38:23.286463 26790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:23.286268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:43231)
0809 16:38:23.286385 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:23.286420 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 16:38:24.110397 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:24.110143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:57963)
0809 16:38:24.110306 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:24.110346 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260809 16:38:24.156210 26792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:24.156005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:42053)
0809 16:38:24.156134 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:24.156166 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260809 16:38:24.287106 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:24.286893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:32939)
0809 16:38:24.287031 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:24.287065 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260809 16:38:25.111104 26794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:25.110852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:35859)
0809 16:38:25.111014 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:25.111052 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260809 16:38:25.156929 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:25.156657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:37773)
0809 16:38:25.156822 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:25.156865 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":81,"threads_started":1}
I20260809 16:38:25.287714 26796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:25.287479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:52269)
0809 16:38:25.287630 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:25.287668 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260809 16:38:26.111716 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:26.111500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:35599)
0809 16:38:26.111629 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:26.111665 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260809 16:38:26.148188 26802 raft_consensus.cc:482] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:26.148285 26802 raft_consensus.cc:504] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15f341b4f0584a16a90d5c506ed9611e" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33001 } } peers { permanent_uuid: "7a12282ec58a4e9eb724fa32f2ee2321" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 42517 } } peers { permanent_uuid: "eb320913384c46359f12a42d710c1063" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 41235 } }
I20260809 16:38:26.148432 26802 leader_election.cc:288] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001), 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517)
W20260809 16:38:26.148772 26120 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 15f341b4f0584a16a90d5c506ed9611e (127.31.15.194:33001): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
W20260809 16:38:26.148859 26118 leader_election.cc:341] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7a12282ec58a4e9eb724fa32f2ee2321 (127.31.15.195:42517): Not found: Tablet not found: dc7432a8e81643168af586426cd5d50b
I20260809 16:38:26.148927 26118 leader_election.cc:302] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb320913384c46359f12a42d710c1063; no voters: 15f341b4f0584a16a90d5c506ed9611e, 7a12282ec58a4e9eb724fa32f2ee2321
I20260809 16:38:26.148996 26802 raft_consensus.cc:2726] T dc7432a8e81643168af586426cd5d50b P eb320913384c46359f12a42d710c1063 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 16:38:26.157621 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:26.157390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:52295)
0809 16:38:26.157536 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:26.157575 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260809 16:38:26.288316 26806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:26.288116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:44019)
0809 16:38:26.288234 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:26.288268 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260809 16:38:27.112413 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:27.112197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.193:36699)
0809 16:38:27.112333 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:27.112366 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260809 16:38:27.158317 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:27.158091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.194:47991)
0809 16:38:27.158230 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:27.158269 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260809 16:38:27.286329 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 26032
I20260809 16:38:27.296938 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:27.296709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:32839 (local address 127.31.15.195:39423)
0809 16:38:27.296842 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:27.296873 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.254:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":54,"threads_started":1}
I20260809 16:38:27.303594 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 26308
I20260809 16:38:27.312707 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 26516
2026-08-09T16:38:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58995 ms)
[----------] 30 tests from SecurityITest (234948 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:27 dist-test-slave-21b9 krb5kdc[26816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:27 dist-test-slave-21b9 krb5kdc[26816](info): set up 2 sockets
Aug 09 16:38:27 dist-test-slave-21b9 krb5kdc[26816](info): commencing operation
krb5kdc: starting...
W20260809 16:38:28.502202 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:28 dist-test-slave-21b9 krb5kdc[26816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:28.613927 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:35739
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:28.694707 26839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:28.694903 26839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:28.694934 26839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:28.696077 26839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:28.696120 26839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:28.696138 26839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:28.696156 26839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:28.696169 26839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:28.697633 26839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:35739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:28.697866 26839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:28.698071 26839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:28.702695 26845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:28.702672 26852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:28.702687 26847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:28.702724 26839 server_base.cc:1034] running on GCE node
I20260809 16:38:28.703090 26839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:28.703421 26839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:28.704556 26839 hybrid_clock.cc:648] HybridClock initialized: now 1786293508704547 us; error 42 us; skew 500 ppm
Aug 09 16:38:28 dist-test-slave-21b9 krb5kdc[26816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293508, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:28.706881 26839 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:28.707351 26839 webserver.cc:460] Webserver started at http://127.31.15.254:42707/ using document root <none> and password file <none>
I20260809 16:38:28.707592 26839 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:28.707636 26839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:28.707762 26839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:28.708843 26839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "daca3d678d1d49368924bb4fab0fd172"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "3ad9f35058f2306b291b094074305106"
server_key_iv: "5de9fa4c721ca14640b922c3f46283dd"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.709183 26839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "daca3d678d1d49368924bb4fab0fd172"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "3ad9f35058f2306b291b094074305106"
server_key_iv: "5de9fa4c721ca14640b922c3f46283dd"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.710438 26839 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:28.711115 26862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:28.711288 26839 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:28.711380 26839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "daca3d678d1d49368924bb4fab0fd172"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "3ad9f35058f2306b291b094074305106"
server_key_iv: "5de9fa4c721ca14640b922c3f46283dd"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.711467 26839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:28.725461 26839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:28.727056 26839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:28.727175 26839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:28.731019 26839 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:35739
I20260809 16:38:28.731060 26946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:35739 every 8 connection(s)
I20260809 16:38:28.731377 26839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:28.732218 26947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:28.734413 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: Bootstrap starting.
I20260809 16:38:28.734938 26947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:28.735174 26947 log.cc:824] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:28.735819 26947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: No bootstrap required, opened a new log
I20260809 16:38:28.737339 26947 raft_consensus.cc:348] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } }
I20260809 16:38:28.737473 26947 raft_consensus.cc:374] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:28.737502 26947 raft_consensus.cc:729] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daca3d678d1d49368924bb4fab0fd172, State: Initialized, Role: FOLLOWER
I20260809 16:38:28.737607 26947 consensus_queue.cc:260] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } }
I20260809 16:38:28.737658 26947 raft_consensus.cc:388] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:28.737676 26947 raft_consensus.cc:482] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:28.737707 26947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:28.738178 26947 raft_consensus.cc:504] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } }
I20260809 16:38:28.738267 26947 leader_election.cc:302] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daca3d678d1d49368924bb4fab0fd172; no voters:
I20260809 16:38:28.738431 26947 leader_election.cc:288] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:28.738453 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 26839
I20260809 16:38:28.738482 26950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:28.738554 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:28.738654 26950 raft_consensus.cc:686] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [term 1 LEADER]: Becoming Leader. State: Replica: daca3d678d1d49368924bb4fab0fd172, State: Running, Role: LEADER
I20260809 16:38:28.738696 26947 sys_catalog.cc:564] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:28.738780 31807 external_mini_cluster.cc:1428] Setting key 10f3d97a72d81a410331236a5e1a7b2c
I20260809 16:38:28.739863 26950 consensus_queue.cc:237] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } }
I20260809 16:38:28.740484 26952 sys_catalog.cc:455] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daca3d678d1d49368924bb4fab0fd172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } } }
I20260809 16:38:28.740597 26951 sys_catalog.cc:455] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader daca3d678d1d49368924bb4fab0fd172. Latest consensus state: current_term: 1 leader_uuid: "daca3d678d1d49368924bb4fab0fd172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daca3d678d1d49368924bb4fab0fd172" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 35739 } } }
I20260809 16:38:28.740908 26952 sys_catalog.cc:458] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:28.740919 26951 sys_catalog.cc:458] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172 [sys.catalog]: This master's current role is: LEADER
W20260809 16:38:28.741252 26973 catalog_manager.cc:1558] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:38:28.741297 26973 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:38:28.741443 26975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:28.741608 26975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:28.743132 26975 catalog_manager.cc:1347] Generated new cluster ID: 2ed918ac62c44eb98d18492b1d2e7cc9
I20260809 16:38:28.743217 26975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:28 dist-test-slave-21b9 krb5kdc[26816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:28.750299 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:28.739241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.31.15.254:35739)
0809 16:38:28.739438 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:28.739446 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:28.739472 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:38:28.739564 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:28.739569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:28.739609 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:28.739716 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:28.740525 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.741890 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.742748 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.743390 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.743506 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:28.746162 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:28.746166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:28.746166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:28.746178 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:28.748707 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:28.749078 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:28.749081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:28.749081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:28.749212 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:28.749390 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:28.749391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:28.749391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:28.749707 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:28.749867 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:28.749946 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 16:38:28.750084 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260809 16:38:28.752084 26975 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:28.752755 26975 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:28.759230 26975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: Generated new TSK 0
I20260809 16:38:28.759438 26975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:38:28.785900 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:28.868696 26983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:28.868896 26983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:28.868918 26983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:28.870060 26983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:28.870103 26983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:28.870128 26983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:28.871613 26983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:28.871845 26983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:28.872056 26983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:28.876693 26992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:28.876783 26989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:28.876786 26988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:28.876858 26983 server_base.cc:1034] running on GCE node
I20260809 16:38:28.877132 26983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:28.877326 26983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:28.878474 26983 hybrid_clock.cc:648] HybridClock initialized: now 1786293508878457 us; error 29 us; skew 500 ppm
Aug 09 16:38:28 dist-test-slave-21b9 krb5kdc[26816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293508, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:28.880940 26983 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:38:28.881345 26983 webserver.cc:460] Webserver started at http://127.31.15.193:42389/ using document root <none> and password file <none>
I20260809 16:38:28.881551 26983 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:28.881592 26983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:28.881695 26983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:28.882642 26983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "b9453ec5fe42443b8de66fb73809ad2d"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "66fbe89c439e3b5498c3c706c6e87e71"
server_key_iv: "aae4b99f10ef126ae169fed1d512d81e"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.882925 26983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "b9453ec5fe42443b8de66fb73809ad2d"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "66fbe89c439e3b5498c3c706c6e87e71"
server_key_iv: "aae4b99f10ef126ae169fed1d512d81e"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.884135 26983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:28.884989 27011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:28.885170 26983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:28.885272 26983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "b9453ec5fe42443b8de66fb73809ad2d"
format_stamp: "Formatted at 2026-08-09 16:38:28 on dist-test-slave-21b9"
server_key: "66fbe89c439e3b5498c3c706c6e87e71"
server_key_iv: "aae4b99f10ef126ae169fed1d512d81e"
server_key_version: "encryptionkey@0"
I20260809 16:38:28.885345 26983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:28.896629 26983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:28.898649 26983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:28.898759 26983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:28.898931 26983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:28.899192 26983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:28.899225 26983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:28.899255 26983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:28.899295 26983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:28.904968 26983 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45027
I20260809 16:38:28.905035 27185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45027 every 8 connection(s)
I20260809 16:38:28.905304 26983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:38:28 dist-test-slave-21b9 krb5kdc[26816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293508, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:28.911245 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 26983
I20260809 16:38:28.911365 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:28.911550 31807 external_mini_cluster.cc:1428] Setting key 4cd1c2b669b4117eb2e9ed2cecc2545b
I20260809 16:38:28.914446 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:28.905945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:42105 (local address 127.31.15.254:35739)
0809 16:38:28.906008 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:28.906010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:28.906176 (+ 166us) server_negotiation.cc:407] Connection header received
0809 16:38:28.906676 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:28.906678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:28.906693 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:28.906744 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:28.907717 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.908229 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.909192 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.909401 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.909456 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:28.911490 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:28.911491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:28.911491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:28.911503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:28.912905 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:28.913330 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:28.913332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:28.913332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:28.913395 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:28.913628 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:28.913629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:28.913629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:28.913754 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:28.913815 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:28.914288 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 16:38:28.914378 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:38:28.914860 27191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:28.905896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:35739 (local address 127.31.15.193:42105)
0809 16:38:28.906066 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:28.906102 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:28.906581 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:28.906811 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:28.906815 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:28.907407 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:28.907630 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:28.907636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.908313 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.908315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:28.909097 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:28.909099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:28.909469 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:28.909470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:28.909600 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:28.910034 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:28.910043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:28.911371 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:28.913006 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:28.913013 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:28.913015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:28.913265 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:28.913457 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:28.913458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:28.913458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:28.913573 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:28.913809 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:28.913813 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:28.913963 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 16:38:28.914536 (+ 573us) client_negotiation.cc:240] Negotiation successful
0809 16:38:28.914702 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260809 16:38:28.915380 27186 heartbeater.cc:347] Connected to a master server at 127.31.15.254:35739
I20260809 16:38:28.915468 27186 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:28.915635 27186 heartbeater.cc:511] Master 127.31.15.254:35739 requested a full tablet report, sending...
I20260809 16:38:28.916168 26893 ts_manager.cc:194] Registered new tserver with Master: b9453ec5fe42443b8de66fb73809ad2d (127.31.15.193:45027)
I20260809 16:38:28.917074 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:42105
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:38:28.937618 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:29.023306 27198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.023602 27198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.023643 27198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.025323 27198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.025389 27198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.025430 27198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:29.027856 27198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.028354 27198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.028630 27198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:29.034345 27208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.034446 27214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.034580 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.034894 27198 server_base.cc:1034] running on GCE node
I20260809 16:38:29.035081 27198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.035329 27198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.036484 27198 hybrid_clock.cc:648] HybridClock initialized: now 1786293509036468 us; error 26 us; skew 500 ppm
Aug 09 16:38:29 dist-test-slave-21b9 krb5kdc[26816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293509, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:29.039486 27198 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:38:29.041934 27198 webserver.cc:460] Webserver started at http://127.31.15.194:40303/ using document root <none> and password file <none>
I20260809 16:38:29.042184 27198 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.042234 27198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.042385 27198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.043876 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "a3c64f5476524e9da64b9e35d10b49c7"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "8efb84e69cd83146731c08b58e409da0"
server_key_iv: "8fbd6e6fe455d0ebb761a5cd5b970abe"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.044288 27198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "a3c64f5476524e9da64b9e35d10b49c7"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "8efb84e69cd83146731c08b58e409da0"
server_key_iv: "8fbd6e6fe455d0ebb761a5cd5b970abe"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.045600 27198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:29.046319 27228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.046456 27198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:29.046571 27198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "a3c64f5476524e9da64b9e35d10b49c7"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "8efb84e69cd83146731c08b58e409da0"
server_key_iv: "8fbd6e6fe455d0ebb761a5cd5b970abe"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.046648 27198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.062243 27198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.064391 27198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.064518 27198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.064747 27198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:29.065047 27198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:29.065081 27198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.065109 27198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:29.065133 27198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.070533 27198 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:41377
I20260809 16:38:29.070597 27393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:41377 every 8 connection(s)
I20260809 16:38:29.070879 27198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:29.072858 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 27198
I20260809 16:38:29.072962 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:29.073163 31807 external_mini_cluster.cc:1428] Setting key a4d1aeccb6f21b6c5936229fa46ab78a
Aug 09 16:38:29 dist-test-slave-21b9 krb5kdc[26816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293509, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:29.080098 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.071578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:35571 (local address 127.31.15.254:35739)
0809 16:38:29.071665 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.071667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.071803 (+ 136us) server_negotiation.cc:407] Connection header received
0809 16:38:29.072266 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.072267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.072277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.072338 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.073304 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.073861 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.074821 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.075022 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.075083 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.077378 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.077380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.077380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:29.077387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.078710 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.079156 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.079157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.079157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.079205 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.079497 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.079497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.079498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.079605 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.079657 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.079958 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.080033 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 16:38:29.080340 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.071531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:35739 (local address 127.31.15.194:35571)
0809 16:38:29.071685 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.071726 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.072193 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.072392 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.072395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.072956 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:29.073178 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.073183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.073947 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.073948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.074743 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.074745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.075098 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.075099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.075232 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.075755 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:29.075765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:29.077278 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:29.078793 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:29.078801 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:29.078803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:29.079086 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:29.079261 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:29.079262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:29.079262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:29.079433 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:29.079668 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:29.079674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:29.079832 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.079950 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.080114 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260809 16:38:29.080830 27396 heartbeater.cc:347] Connected to a master server at 127.31.15.254:35739
I20260809 16:38:29.080920 27396 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:29.081091 27396 heartbeater.cc:511] Master 127.31.15.254:35739 requested a full tablet report, sending...
I20260809 16:38:29.081468 26893 ts_manager.cc:194] Registered new tserver with Master: a3c64f5476524e9da64b9e35d10b49c7 (127.31.15.194:41377)
I20260809 16:38:29.081842 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:35571
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:38:29.097813 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:29.178301 27404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.178507 27404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.178539 27404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.179661 27404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.179700 27404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.179734 27404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:29.181157 27404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:35739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.181388 27404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.181589 27404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 16:38:29.186337 27404 server_base.cc:1034] running on GCE node
W20260809 16:38:29.186287 27413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.186254 27416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.186259 27410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.186741 27404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.186976 27404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.188098 27404 hybrid_clock.cc:648] HybridClock initialized: now 1786293509188082 us; error 26 us; skew 500 ppm
Aug 09 16:38:29 dist-test-slave-21b9 krb5kdc[26816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293509, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:29.190431 27404 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:38:29.190824 27404 webserver.cc:460] Webserver started at http://127.31.15.195:39623/ using document root <none> and password file <none>
I20260809 16:38:29.191031 27404 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.191073 27404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.191191 27404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.192210 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "f24e09545e7649e58d13761463611898"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "56cf9e0ecc1e7a2086e144fa88a57355"
server_key_iv: "260f1b3f29ecfb80baf73623aa8e319e"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.192524 27404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "f24e09545e7649e58d13761463611898"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "56cf9e0ecc1e7a2086e144fa88a57355"
server_key_iv: "260f1b3f29ecfb80baf73623aa8e319e"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.193782 27404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:29.194427 27425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.194576 27404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:29.194658 27404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "f24e09545e7649e58d13761463611898"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "56cf9e0ecc1e7a2086e144fa88a57355"
server_key_iv: "260f1b3f29ecfb80baf73623aa8e319e"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.194716 27404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.204311 27404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.206079 27404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.206205 27404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.206394 27404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:29.206681 27404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:29.206720 27404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.206748 27404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:29.206772 27404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.212251 27404 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:39975
I20260809 16:38:29.212308 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:39975 every 8 connection(s)
I20260809 16:38:29.212586 27404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:38:29 dist-test-slave-21b9 krb5kdc[26816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293509, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:29.220800 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.213225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:54559 (local address 127.31.15.254:35739)
0809 16:38:29.213301 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.213303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.213433 (+ 130us) server_negotiation.cc:407] Connection header received
0809 16:38:29.213950 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.213951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.213969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.214008 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.215089 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.215638 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.216557 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.216737 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.216783 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.218686 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.218687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.218687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:29.218694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.219798 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.220136 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.220137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.220137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.220182 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.220355 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.220355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.220355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.220442 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.220489 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.220681 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.220736 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 16:38:29.220966 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.213188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:35739 (local address 127.31.15.195:54559)
0809 16:38:29.213323 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.213358 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.213856 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.214125 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.214129 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.214818 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:29.215012 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.215017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.215716 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.215717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.216493 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.216496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.216792 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.216792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.216886 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.217326 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:29.217334 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:29.218609 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:29.219862 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:29.219868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:29.219870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:29.220069 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:29.220225 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:29.220226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:29.220226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:29.220314 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:29.220486 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:29.220489 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:29.220588 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.220662 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.220771 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260809 16:38:29.221393 27618 heartbeater.cc:347] Connected to a master server at 127.31.15.254:35739
I20260809 16:38:29.221491 27618 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:29.221657 27618 heartbeater.cc:511] Master 127.31.15.254:35739 requested a full tablet report, sending...
I20260809 16:38:29.222002 26893 ts_manager.cc:194] Registered new tserver with Master: f24e09545e7649e58d13761463611898 (127.31.15.195:39975)
I20260809 16:38:29.222333 26893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:54559
I20260809 16:38:29.222697 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 27404
I20260809 16:38:29.222823 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:29.223045 31807 external_mini_cluster.cc:1428] Setting key 7ce5b424e634500aaccb6ed0a28f597f
I20260809 16:38:29.223762 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:29.229847 26959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.224982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.31.15.254:35739)
0809 16:38:29.225041 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.225043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.225126 (+ 83us) server_negotiation.cc:407] Connection header received
0809 16:38:29.225165 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.225165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.225173 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.225213 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.225699 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.226133 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.226763 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.226925 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.226968 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.227980 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.227981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.227981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:29.227986 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.229008 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.229287 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.229287 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.229287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.229328 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:29.229478 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:29.229478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:29.229478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:29.229596 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.229666 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.229740 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.229791 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:38:29.231216 26893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:29.232752 26893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:29.238729 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:41377 (local address 127.0.0.1:53542)
0809 16:38:29.234987 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.235028 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.235093 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.235476 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.235480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.235483 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.235547 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.235549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236481 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.236482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.237599 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.237602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238367 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.238466 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238528 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.238597 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.238657 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":64,"threads_started":1}
I20260809 16:38:29.238746 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.31.15.194:41377)
0809 16:38:29.235073 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.235077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.235088 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:38:29.235213 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.235215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.235230 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.235340 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.235644 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236380 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.237692 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238297 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238355 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238414 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.238619 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.238665 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":90,"threads_started":1}
I20260809 16:38:29.238976 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39975 (local address 127.0.0.1:52468)
0809 16:38:29.235124 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.235133 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.235207 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.235493 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.235495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.235496 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.235567 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.235569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236866 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.236867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.237729 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.237731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238481 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.238561 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238564 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.238607 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.238917 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":303,"thread_start_us":50,"threads_started":1}
I20260809 16:38:29.239135 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45027 (local address 127.0.0.1:42338)
0809 16:38:29.235154 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.235164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.235220 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.235556 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.235557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.235558 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.235589 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.235590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236857 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.236858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.237970 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.237972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238932 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.238979 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238983 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.239027 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.239064 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":28,"thread_start_us":75,"threads_started":1}
I20260809 16:38:29.239168 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.31.15.193:45027)
0809 16:38:29.235146 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.235150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.235207 (+ 57us) server_negotiation.cc:407] Connection header received
0809 16:38:29.235398 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.235400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.235415 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.235478 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.235658 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236754 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238049 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238785 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238849 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238901 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.239055 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.239094 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260809 16:38:29.239800 27301 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8b22649080453fb9a4cd58598fa06b (DEFAULT_TABLE table=test-table [id=5ba9f6c2d746496a90926f0b45761cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.240142 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8b22649080453fb9a4cd58598fa06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.242705 27646 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Bootstrap starting.
I20260809 16:38:29.238765 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.234831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.31.15.195:39975)
0809 16:38:29.234992 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.234997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.235182 (+ 185us) server_negotiation.cc:407] Connection header received
0809 16:38:29.235312 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.235316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.235332 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.235402 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.235652 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.236753 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.237799 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.238402 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.238467 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.238519 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.238651 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.238694 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260809 16:38:29.243425 27646 tablet_bootstrap.cc:654] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.243561 27091 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8b22649080453fb9a4cd58598fa06b (DEFAULT_TABLE table=test-table [id=5ba9f6c2d746496a90926f0b45761cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.243669 27506 tablet_service.cc:1467] Processing CreateTablet for tablet 7b8b22649080453fb9a4cd58598fa06b (DEFAULT_TABLE table=test-table [id=5ba9f6c2d746496a90926f0b45761cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.243918 27091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8b22649080453fb9a4cd58598fa06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.243975 27506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b8b22649080453fb9a4cd58598fa06b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.244009 27646 log.cc:824] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.244802 27646 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: No bootstrap required, opened a new log
I20260809 16:38:29.244861 27646 ts_tablet_manager.cc:1397] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:38:29.246420 27650 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Bootstrap starting.
I20260809 16:38:29.246770 27646 raft_consensus.cc:348] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.246955 27646 raft_consensus.cc:374] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.247031 27646 raft_consensus.cc:729] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c64f5476524e9da64b9e35d10b49c7, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.247089 27650 tablet_bootstrap.cc:654] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.247226 27646 consensus_queue.cc:260] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.247308 27651 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: Bootstrap starting.
I20260809 16:38:29.247536 27646 ts_tablet_manager.cc:1428] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:38:29.247646 27396 heartbeater.cc:503] Master 127.31.15.254:35739 was elected leader, sending a full tablet report...
I20260809 16:38:29.247709 27650 log.cc:824] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.247988 27651 tablet_bootstrap.cc:654] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.248505 27650 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: No bootstrap required, opened a new log
I20260809 16:38:29.248519 27651 log.cc:824] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.248626 27650 ts_tablet_manager.cc:1397] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:38:29.249215 27651 tablet_bootstrap.cc:492] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: No bootstrap required, opened a new log
I20260809 16:38:29.249274 27651 ts_tablet_manager.cc:1397] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:29.250260 27650 raft_consensus.cc:348] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.250428 27650 raft_consensus.cc:374] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.250458 27650 raft_consensus.cc:729] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24e09545e7649e58d13761463611898, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.250566 27650 consensus_queue.cc:260] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.250808 27650 ts_tablet_manager.cc:1428] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:38:29.250833 27618 heartbeater.cc:503] Master 127.31.15.254:35739 was elected leader, sending a full tablet report...
I20260809 16:38:29.250828 27651 raft_consensus.cc:348] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.251009 27651 raft_consensus.cc:374] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.251055 27651 raft_consensus.cc:729] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9453ec5fe42443b8de66fb73809ad2d, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.251164 27651 consensus_queue.cc:260] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.251466 27651 ts_tablet_manager.cc:1428] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 16:38:29.251585 27186 heartbeater.cc:503] Master 127.31.15.254:35739 was elected leader, sending a full tablet report...
W20260809 16:38:29.321406 27397 tablet.cc:2369] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:29.338757 27652 raft_consensus.cc:482] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:29.338886 27652 raft_consensus.cc:504] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.339215 27652 leader_election.cc:288] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9453ec5fe42443b8de66fb73809ad2d (127.31.15.193:45027), f24e09545e7649e58d13761463611898 (127.31.15.195:39975)
I20260809 16:38:29.343253 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.339509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:54213 (local address 127.31.15.195:39975)
0809 16:38:29.339553 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.339555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.339666 (+ 111us) server_negotiation.cc:407] Connection header received
0809 16:38:29.339695 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.339696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.339705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.339773 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.339899 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.340632 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342226 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.342828 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342912 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.342921 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.343137 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.343177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:38:29.343253 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.339470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:34841 (local address 127.31.15.193:45027)
0809 16:38:29.339511 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.339513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.339522 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:38:29.339613 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.339615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.339624 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.339687 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.339874 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.340647 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342226 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.342844 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342912 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.342921 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.343137 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.343181 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:38:29.343289 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.339408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45027 (local address 127.31.15.194:34841)
0809 16:38:29.339476 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.339487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.339532 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.339697 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.339699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.339700 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.339802 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.339804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.340735 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.340736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.342112 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.342117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.342941 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.343059 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.343061 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.343122 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.343172 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":84}
I20260809 16:38:29.343299 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.339412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39975 (local address 127.31.15.194:54213)
0809 16:38:29.339596 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.339605 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.339654 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.339785 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.339786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.339787 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.339838 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.339839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.340735 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.340736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.342112 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.342117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.342942 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.342943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.343055 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.343061 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.343122 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.343171 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":84,"thread_start_us":104,"threads_started":1}
I20260809 16:38:29.343739 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8b22649080453fb9a4cd58598fa06b" candidate_uuid: "a3c64f5476524e9da64b9e35d10b49c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9453ec5fe42443b8de66fb73809ad2d" is_pre_election: true
I20260809 16:38:29.343894 27121 raft_consensus.cc:2454] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c64f5476524e9da64b9e35d10b49c7 in term 0.
I20260809 16:38:29.343897 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8b22649080453fb9a4cd58598fa06b" candidate_uuid: "a3c64f5476524e9da64b9e35d10b49c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24e09545e7649e58d13761463611898" is_pre_election: true
I20260809 16:38:29.344058 27544 raft_consensus.cc:2454] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3c64f5476524e9da64b9e35d10b49c7 in term 0.
I20260809 16:38:29.344144 27229 leader_election.cc:302] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c64f5476524e9da64b9e35d10b49c7, b9453ec5fe42443b8de66fb73809ad2d; no voters:
I20260809 16:38:29.344277 27652 raft_consensus.cc:2781] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:29.344336 27652 raft_consensus.cc:482] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:29.344362 27652 raft_consensus.cc:3037] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:29.345080 27652 raft_consensus.cc:504] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.345206 27652 leader_election.cc:288] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [CANDIDATE]: Term 1 election: Requested vote from peers b9453ec5fe42443b8de66fb73809ad2d (127.31.15.193:45027), f24e09545e7649e58d13761463611898 (127.31.15.195:39975)
I20260809 16:38:29.345345 27544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8b22649080453fb9a4cd58598fa06b" candidate_uuid: "a3c64f5476524e9da64b9e35d10b49c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24e09545e7649e58d13761463611898"
I20260809 16:38:29.345366 27121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b8b22649080453fb9a4cd58598fa06b" candidate_uuid: "a3c64f5476524e9da64b9e35d10b49c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9453ec5fe42443b8de66fb73809ad2d"
I20260809 16:38:29.345425 27544 raft_consensus.cc:3037] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:29.345434 27121 raft_consensus.cc:3037] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:29.346119 27121 raft_consensus.cc:2454] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c64f5476524e9da64b9e35d10b49c7 in term 1.
I20260809 16:38:29.346171 27544 raft_consensus.cc:2454] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3c64f5476524e9da64b9e35d10b49c7 in term 1.
I20260809 16:38:29.346283 27229 leader_election.cc:302] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3c64f5476524e9da64b9e35d10b49c7, b9453ec5fe42443b8de66fb73809ad2d; no voters:
I20260809 16:38:29.346386 27652 raft_consensus.cc:2781] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:29.346568 27652 raft_consensus.cc:686] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 1 LEADER]: Becoming Leader. State: Replica: a3c64f5476524e9da64b9e35d10b49c7, State: Running, Role: LEADER
I20260809 16:38:29.346663 27652 consensus_queue.cc:237] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } }
I20260809 16:38:29.347484 26893 catalog_manager.cc:5495] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3c64f5476524e9da64b9e35d10b49c7 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "a3c64f5476524e9da64b9e35d10b49c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3c64f5476524e9da64b9e35d10b49c7" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 41377 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 } health_report { overall_health: UNKNOWN } } }
I20260809 16:38:29.358273 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.356191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53550 (local address 127.31.15.194:41377)
0809 16:38:29.356240 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.356241 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.356249 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:38:29.356310 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.356311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.356326 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.356382 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:29.356490 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.356942 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.357618 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.357788 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.357829 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.358004 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:29.358106 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:29.358159 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.358206 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.358222 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:38:29.360057 27121 raft_consensus.cc:1261] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 1 FOLLOWER]: Refusing update from remote peer a3c64f5476524e9da64b9e35d10b49c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:29.360071 27544 raft_consensus.cc:1261] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 1 FOLLOWER]: Refusing update from remote peer a3c64f5476524e9da64b9e35d10b49c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:29.360405 27652 consensus_queue.cc:1035] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24e09545e7649e58d13761463611898" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:29.360535 27663 consensus_queue.cc:1035] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9453ec5fe42443b8de66fb73809ad2d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:29.363021 27668 mvcc.cc:205] Tried to move back new op lower bound from 7316658214335905792 to 7316658214284533760. Current Snapshot: MvccSnapshot[applied={T|T < 7316658214335905792}]
I20260809 16:38:29.365185 26890 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:29.365267 26890 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:29.365970 26890 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daca3d678d1d49368924bb4fab0fd172: Sending DeleteTablet for 3 replicas of tablet 7b8b22649080453fb9a4cd58598fa06b
I20260809 16:38:29.366325 27301 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8b22649080453fb9a4cd58598fa06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:29 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:53542
I20260809 16:38:29.366320 27506 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8b22649080453fb9a4cd58598fa06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:29 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:52468
I20260809 16:38:29.366345 27091 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b8b22649080453fb9a4cd58598fa06b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:29 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:42338
I20260809 16:38:29.366573 27686 tablet_replica.cc:330] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d: stopping tablet replica
I20260809 16:38:29.366670 27686 raft_consensus.cc:2229] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:29.366813 27686 raft_consensus.cc:2258] T 7b8b22649080453fb9a4cd58598fa06b P b9453ec5fe42443b8de66fb73809ad2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:29.366815 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 26983
I20260809 16:38:29.375056 27683 tablet_replica.cc:330] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: stopping tablet replica
I20260809 16:38:29.375173 27683 raft_consensus.cc:2229] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:38:29.375329 27683 raft_consensus.cc:2258] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:29.375520 27685 tablet_replica.cc:330] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: stopping tablet replica
I20260809 16:38:29.375660 27685 raft_consensus.cc:2229] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:29.375797 27685 raft_consensus.cc:2258] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:29.376482 27683 ts_tablet_manager.cc:1905] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:29.378170 27683 ts_tablet_manager.cc:1918] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:29.378247 27683 log.cc:1196] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/7b8b22649080453fb9a4cd58598fa06b
I20260809 16:38:29.378477 27683 ts_tablet_manager.cc:1939] T 7b8b22649080453fb9a4cd58598fa06b P a3c64f5476524e9da64b9e35d10b49c7: Deleting consensus metadata
I20260809 16:38:29.378901 26864 catalog_manager.cc:4841] TS a3c64f5476524e9da64b9e35d10b49c7 (127.31.15.194:41377): tablet 7b8b22649080453fb9a4cd58598fa06b (table test-table [id=5ba9f6c2d746496a90926f0b45761cf1]) successfully deleted
I20260809 16:38:29.379616 27685 ts_tablet_manager.cc:1905] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 16:38:29.380421 26864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:45027 (error 108)
I20260809 16:38:29.380689 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 27198
I20260809 16:38:29.381135 27685 ts_tablet_manager.cc:1918] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:29.381250 27685 log.cc:1196] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/7b8b22649080453fb9a4cd58598fa06b
I20260809 16:38:29.381603 27685 ts_tablet_manager.cc:1939] T 7b8b22649080453fb9a4cd58598fa06b P f24e09545e7649e58d13761463611898: Deleting consensus metadata
I20260809 16:38:29.382937 26866 catalog_manager.cc:4841] TS f24e09545e7649e58d13761463611898 (127.31.15.195:39975): tablet 7b8b22649080453fb9a4cd58598fa06b (table test-table [id=5ba9f6c2d746496a90926f0b45761cf1]) successfully deleted
I20260809 16:38:29.383241 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.383099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45027 (local address 127.0.0.1:42344)
0809 16:38:29.383160 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.383206 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 16:38:29.383816 26864 catalog_manager.cc:4584] TS b9453ec5fe42443b8de66fb73809ad2d (127.31.15.193:45027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b8b22649080453fb9a4cd58598fa06b: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45027: connect: Connection refused (error 111)
I20260809 16:38:29.390594 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 27404
I20260809 16:38:29.402114 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 26839
2026-08-09T16:38:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2064 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T16:38:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:29Z Disabled control of system clock
I20260809 16:38:29.430413 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45637
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:45637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:29.510850 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.511072 27693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.511101 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.512290 27693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:29.512336 27693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.512351 27693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.512367 27693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:29.512382 27693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:29.513725 27693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:45637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:45637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.513968 27693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.514247 27693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:29.518893 27701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.518859 27699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.518878 27698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.519155 27693 server_base.cc:1034] running on GCE node
I20260809 16:38:29.519403 27693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.519668 27693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.520805 27693 hybrid_clock.cc:648] HybridClock initialized: now 1786293509520790 us; error 29 us; skew 500 ppm
I20260809 16:38:29.521976 27693 webserver.cc:460] Webserver started at http://127.31.15.254:37981/ using document root <none> and password file <none>
I20260809 16:38:29.522224 27693 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.522274 27693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.522405 27693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.523473 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "aedfa7d8158a499e83f9541485f7f941"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "96d6ff47980d7dbd28f43b7eb059f027"
server_key_iv: "057d7c98414f66705db6b4c7e0cd5bbd"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.523797 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "aedfa7d8158a499e83f9541485f7f941"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "96d6ff47980d7dbd28f43b7eb059f027"
server_key_iv: "057d7c98414f66705db6b4c7e0cd5bbd"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.525105 27693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:29.525928 27710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.526117 27693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:29.526216 27693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "aedfa7d8158a499e83f9541485f7f941"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "96d6ff47980d7dbd28f43b7eb059f027"
server_key_iv: "057d7c98414f66705db6b4c7e0cd5bbd"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.526284 27693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.543195 27693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.543495 27693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.543622 27693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.547326 27693 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:45637
I20260809 16:38:29.547375 27792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:45637 every 8 connection(s)
I20260809 16:38:29.547683 27693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:29.548630 27794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.550835 27794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: Bootstrap starting.
I20260809 16:38:29.551409 27794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.551692 27794 log.cc:824] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.552337 27794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: No bootstrap required, opened a new log
I20260809 16:38:29.553894 27794 raft_consensus.cc:348] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } }
I20260809 16:38:29.554028 27794 raft_consensus.cc:374] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.554049 27794 raft_consensus.cc:729] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aedfa7d8158a499e83f9541485f7f941, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.554220 27794 consensus_queue.cc:260] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } }
I20260809 16:38:29.554320 27797 raft_consensus.cc:482] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:29.554414 27797 raft_consensus.cc:504] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } }
I20260809 16:38:29.554306 27794 raft_consensus.cc:388] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:29.554566 27794 raft_consensus.cc:482] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:29.554543 27797 leader_election.cc:302] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aedfa7d8158a499e83f9541485f7f941; no voters:
I20260809 16:38:29.554625 27794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:29.554682 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 27693
I20260809 16:38:29.554744 27797 leader_election.cc:288] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260809 16:38:29.554797 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:29.554998 31807 external_mini_cluster.cc:1428] Setting key bcfcd56db227579702de11549a73da0d
I20260809 16:38:29.555305 27794 raft_consensus.cc:504] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } }
I20260809 16:38:29.555382 27794 leader_election.cc:302] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aedfa7d8158a499e83f9541485f7f941; no voters:
I20260809 16:38:29.555423 27794 leader_election.cc:288] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:29.555482 27798 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:29.555665 27798 raft_consensus.cc:686] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 1 LEADER]: Becoming Leader. State: Replica: aedfa7d8158a499e83f9541485f7f941, State: Running, Role: LEADER
I20260809 16:38:29.555670 27794 sys_catalog.cc:564] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:29.555779 27798 consensus_queue.cc:237] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } }
I20260809 16:38:29.556114 27797 raft_consensus.cc:2741] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [term 1 LEADER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 16:38:29.556264 27800 sys_catalog.cc:455] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aedfa7d8158a499e83f9541485f7f941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } } }
I20260809 16:38:29.556349 27800 sys_catalog.cc:458] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:29.556510 27801 sys_catalog.cc:455] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aedfa7d8158a499e83f9541485f7f941. Latest consensus state: current_term: 1 leader_uuid: "aedfa7d8158a499e83f9541485f7f941" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aedfa7d8158a499e83f9541485f7f941" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 45637 } } }
I20260809 16:38:29.556571 27801 sys_catalog.cc:458] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:29.556550 27823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:29.557055 27823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:29.559919 27823 catalog_manager.cc:1347] Generated new cluster ID: 8501be8e45b0415691ca218a2a649ec1
I20260809 16:38:29.559978 27823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:29.572832 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.556011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49750 (local address 127.31.15.254:45637)
0809 16:38:29.556442 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.556454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.556490 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:38:29.556605 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.556613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.556682 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.556817 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.557144 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.558212 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.559242 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.559717 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.559785 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.572078 (+ 12293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.572083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.572083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:29.572305 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.572353 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:29.572368 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.572444 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.572512 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.572603 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260809 16:38:29.577204 27823 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:29.577786 27823 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:29.581553 27823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: Generated new TSK 0
I20260809 16:38:29.581717 27823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:29.588837 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--builtin_ntp_servers=127.31.15.212:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:29.671029 27841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.671231 27841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.671252 27841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.672420 27841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.672461 27841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.672484 27841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:29.673823 27841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.674052 27841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.674341 27841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:29.679080 27847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.679095 27848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.679128 27841 server_base.cc:1034] running on GCE node
W20260809 16:38:29.679095 27851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.679507 27841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.679713 27841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.680850 27841 hybrid_clock.cc:648] HybridClock initialized: now 1786293509680840 us; error 29 us; skew 500 ppm
I20260809 16:38:29.681954 27841 webserver.cc:460] Webserver started at http://127.31.15.193:42491/ using document root <none> and password file <none>
I20260809 16:38:29.682169 27841 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.682214 27841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.682332 27841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.683362 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "f54ff807831e5b72ab250f1bb49dc43a"
server_key_iv: "f123336a40c889c02f85b19b3faf2947"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.683697 27841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "f54ff807831e5b72ab250f1bb49dc43a"
server_key_iv: "f123336a40c889c02f85b19b3faf2947"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.684894 27841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 16:38:29.685514 27859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.685660 27841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:29.685738 27841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "f54ff807831e5b72ab250f1bb49dc43a"
server_key_iv: "f123336a40c889c02f85b19b3faf2947"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.685791 27841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.704120 27841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.704433 27841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.704557 27841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.704761 27841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:29.705077 27841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:29.705114 27841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.705138 27841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:29.705157 27841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.711184 27841 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:46051
I20260809 16:38:29.711251 28042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:46051 every 8 connection(s)
I20260809 16:38:29.711609 27841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:38:29.714453 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 27841
I20260809 16:38:29.714573 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:29.714807 31807 external_mini_cluster.cc:1428] Setting key df65d22da9347158810f25319eb7ee10
I20260809 16:38:29.715922 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--builtin_ntp_servers=127.31.15.212:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:38:29.717221 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.712377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:46965 (local address 127.31.15.254:45637)
0809 16:38:29.712467 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.712469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.712649 (+ 180us) server_negotiation.cc:407] Connection header received
0809 16:38:29.713157 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.713159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.713169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.713220 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.713864 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.714382 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.715796 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.715995 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.716059 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.716962 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.716963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.716964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:29.716968 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.716989 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:29.716993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.717052 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.717109 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.717142 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 16:38:29.717417 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.712334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45637 (local address 127.31.15.193:46965)
0809 16:38:29.712525 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.712561 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.713076 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.713278 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.713281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.713289 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:29.713791 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.713796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.714477 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.714478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.715695 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.715699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.716133 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.716134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.716248 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.716814 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:29.716814 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:29.716843 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:29.716867 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:29.717054 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:29.717056 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:29.717058 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.717108 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.717187 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":102,"threads_started":1}
I20260809 16:38:29.718096 28043 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45637
I20260809 16:38:29.718196 28043 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:29.718384 28043 heartbeater.cc:511] Master 127.31.15.254:45637 requested a full tablet report, sending...
I20260809 16:38:29.718923 27736 ts_manager.cc:194] Registered new tserver with Master: 5acbc8f01e974aecb0a22bfa3ae8ad16 (127.31.15.193:46051)
I20260809 16:38:29.719815 27736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.193:46965
W20260809 16:38:29.802012 28052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.802205 28052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.802227 28052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.803399 28052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.803444 28052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.803470 28052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:29.804781 28052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.804968 28052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.805214 28052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:29.809864 28058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.809880 28059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.809923 28052 server_base.cc:1034] running on GCE node
W20260809 16:38:29.809906 28061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.810237 28052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.810448 28052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.811610 28052 hybrid_clock.cc:648] HybridClock initialized: now 1786293509811588 us; error 36 us; skew 500 ppm
I20260809 16:38:29.812786 28052 webserver.cc:460] Webserver started at http://127.31.15.194:40533/ using document root <none> and password file <none>
I20260809 16:38:29.812994 28052 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.813038 28052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.813164 28052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.814162 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "b8d9c4e9d16b43bca40faab1304fcd50"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "302442f051818a22b2584d251ba60a37"
server_key_iv: "fdd1a69e3dcb3a69d6880c7c6b9a3451"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.814486 28052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "b8d9c4e9d16b43bca40faab1304fcd50"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "302442f051818a22b2584d251ba60a37"
server_key_iv: "fdd1a69e3dcb3a69d6880c7c6b9a3451"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.815768 28052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:29.816452 28068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.816653 28052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:29.816730 28052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "b8d9c4e9d16b43bca40faab1304fcd50"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "302442f051818a22b2584d251ba60a37"
server_key_iv: "fdd1a69e3dcb3a69d6880c7c6b9a3451"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.816779 28052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.829159 28052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.829378 28052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.829471 28052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.829622 28052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:29.829859 28052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:29.829887 28052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.829919 28052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:29.829937 28052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.835225 28052 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:33943
I20260809 16:38:29.835304 28244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:33943 every 8 connection(s)
I20260809 16:38:29.835583 28052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:29.839931 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.836188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:56645 (local address 127.31.15.254:45637)
0809 16:38:29.836264 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.836266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.836447 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:38:29.836884 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.836885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.836899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.836933 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.837512 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.837985 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.838839 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.839021 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.839062 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.839738 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.839739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.839740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:29.839744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.839757 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:29.839761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.839798 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.839844 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.839871 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 16:38:29.840106 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.836168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45637 (local address 127.31.15.194:56645)
0809 16:38:29.836337 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.836368 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.836801 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.836989 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.836992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.837000 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:29.837451 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.837454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.838044 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.838044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.838780 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.838782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.839078 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.839079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.839185 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.839621 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:29.839622 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:29.839647 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:29.839667 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:29.839799 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:29.839801 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:29.839803 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.839844 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.839925 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260809 16:38:29.840540 28246 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45637
I20260809 16:38:29.840620 28246 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:29.840786 28246 heartbeater.cc:511] Master 127.31.15.254:45637 requested a full tablet report, sending...
I20260809 16:38:29.841146 27736 ts_manager.cc:194] Registered new tserver with Master: b8d9c4e9d16b43bca40faab1304fcd50 (127.31.15.194:33943)
I20260809 16:38:29.841511 27736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.194:56645
I20260809 16:38:29.842214 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 28052
I20260809 16:38:29.842303 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:29.842479 31807 external_mini_cluster.cc:1428] Setting key 1a0e68da7baba0089872670f318c201d
I20260809 16:38:29.843338 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--builtin_ntp_servers=127.31.15.212:38645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:29.923498 28249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:29.923689 28249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:29.923718 28249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:29.924840 28249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:29.924880 28249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:29.924911 28249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:29.926224 28249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:45637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:29.926441 28249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:29.926712 28249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:29.931399 28260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.931442 28249 server_base.cc:1034] running on GCE node
W20260809 16:38:29.931432 28257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:29.931416 28256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:29.931802 28249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:29.932030 28249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:29.933178 28249 hybrid_clock.cc:648] HybridClock initialized: now 1786293509933167 us; error 30 us; skew 500 ppm
I20260809 16:38:29.934358 28249 webserver.cc:460] Webserver started at http://127.31.15.195:41065/ using document root <none> and password file <none>
I20260809 16:38:29.934571 28249 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:29.934615 28249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:29.934741 28249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:29.935797 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "276f6ad754a74046a1e9a171d4160a7b"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "5afec386190372618f309e5201db85a9"
server_key_iv: "b92d6f9e8215205e6d6d1bdc2e81567a"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.936125 28249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "276f6ad754a74046a1e9a171d4160a7b"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "5afec386190372618f309e5201db85a9"
server_key_iv: "b92d6f9e8215205e6d6d1bdc2e81567a"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.937284 28249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:29.937927 28268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.938074 28249 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:29.938156 28249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "276f6ad754a74046a1e9a171d4160a7b"
format_stamp: "Formatted at 2026-08-09 16:38:29 on dist-test-slave-21b9"
server_key: "5afec386190372618f309e5201db85a9"
server_key_iv: "b92d6f9e8215205e6d6d1bdc2e81567a"
server_key_version: "encryptionkey@0"
I20260809 16:38:29.938217 28249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:29.952862 28249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:29.953120 28249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:29.953238 28249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:29.953428 28249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:29.953720 28249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:29.953752 28249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.953780 28249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:29.953804 28249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:29.959388 28249 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:41681
I20260809 16:38:29.959448 28444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:41681 every 8 connection(s)
I20260809 16:38:29.959738 28249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:38:29.964354 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.960396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:33575 (local address 127.31.15.254:45637)
0809 16:38:29.960446 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.960447 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.960650 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:38:29.961102 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.961103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.961112 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.961162 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.961763 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.962254 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.963188 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.963395 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.963456 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.964177 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.964177 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.964178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:29.964181 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.964191 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:29.964193 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.964221 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.964278 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.964296 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:38:29.964540 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.960367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:45637 (local address 127.31.15.195:33575)
0809 16:38:29.960535 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.960573 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.961023 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.961221 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.961224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.961233 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:29.961698 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.961702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.962323 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.962324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.963098 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.963100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.963460 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.963460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.963574 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.964065 (+ 491us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:29.964066 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:29.964094 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:29.964117 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:29.964228 (+ 111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:29.964230 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:29.964233 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.964275 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.964351 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260809 16:38:29.964973 28447 heartbeater.cc:347] Connected to a master server at 127.31.15.254:45637
I20260809 16:38:29.965056 28447 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:29.965224 28447 heartbeater.cc:511] Master 127.31.15.254:45637 requested a full tablet report, sending...
I20260809 16:38:29.965555 27736 ts_manager.cc:194] Registered new tserver with Master: 276f6ad754a74046a1e9a171d4160a7b (127.31.15.195:41681)
I20260809 16:38:29.965890 27736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.195:33575
I20260809 16:38:29.968393 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 28249
I20260809 16:38:29.968487 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:29.968673 31807 external_mini_cluster.cc:1428] Setting key 70d4e9ac3329584ba51ab4782bf1af83
I20260809 16:38:29.969515 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:29.973135 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.970754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.31.15.254:45637)
0809 16:38:29.970810 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.970812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.970938 (+ 126us) server_negotiation.cc:407] Connection header received
0809 16:38:29.970972 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.970973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.970981 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.971023 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:29.971146 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.971624 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.972277 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.972442 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.972481 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.972968 (+ 487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:29.972969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:29.972969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:29.972973 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:29.972984 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:29.972987 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:29.973024 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.973064 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.973079 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 16:38:29.974378 27736 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:29.975878 27736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:29.982105 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.978318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.31.15.195:41681)
0809 16:38:29.978546 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.978550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.978558 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:38:29.978590 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.978592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.978606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.978808 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.978852 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.979555 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981053 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.981752 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981823 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.981910 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.981997 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.982032 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":63,"threads_started":1}
I20260809 16:38:29.982137 28466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.978154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:41681 (local address 127.0.0.1:48830)
0809 16:38:29.978248 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.978255 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.978291 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.978698 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.978699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.978701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.978739 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.978740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.979626 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.979627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.980973 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.980974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.981824 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.981900 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.981914 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.981974 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.982050 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":66,"thread_start_us":63,"threads_started":1}
I20260809 16:38:29.982247 28464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.977984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46051 (local address 127.0.0.1:59608)
0809 16:38:29.978105 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.978113 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.978148 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.978374 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.978375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.978378 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.978428 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.978430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.979734 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.979735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.980962 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.980965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.981840 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.981936 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.981940 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.981973 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.982040 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":9,"thread_start_us":87,"threads_started":1}
I20260809 16:38:29.982336 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.978077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.31.15.193:46051)
0809 16:38:29.978244 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.978247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.978255 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:38:29.978291 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.978293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.978306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.978377 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.978482 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.979650 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981048 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.981752 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.982099 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.982200 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.982225 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.982261 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260809 16:38:29.983067 28345 tablet_service.cc:1467] Processing CreateTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 (DEFAULT_TABLE table=test-table [id=e45174fa31d84f72b0c6bc9480712da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.983119 27930 tablet_service.cc:1467] Processing CreateTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 (DEFAULT_TABLE table=test-table [id=e45174fa31d84f72b0c6bc9480712da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.983350 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.977787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33943 (local address 127.0.0.1:57046)
0809 16:38:29.977894 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:29.977934 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:29.977988 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:29.978332 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:29.978336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:29.978339 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:29.978421 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.978424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.981061 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.981708 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:29.981709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.983100 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.983101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:29.983162 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.983166 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:29.983229 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 16:38:29.983289 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260809 16:38:29.983393 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:29.977877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57046 (local address 127.31.15.194:33943)
0809 16:38:29.978138 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:29.978143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:29.978154 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:38:29.978195 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:29.978197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:29.978212 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:29.978506 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:29.978573 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.979656 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.981788 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:29.983011 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:29.983084 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:29.983168 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:29.983246 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 16:38:29.983316 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":84,"threads_started":1}
I20260809 16:38:29.983494 28345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2263597d8dc742ed8a35eccb7c570f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.983506 27930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2263597d8dc742ed8a35eccb7c570f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.983952 28146 tablet_service.cc:1467] Processing CreateTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 (DEFAULT_TABLE table=test-table [id=e45174fa31d84f72b0c6bc9480712da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:29.984285 28146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2263597d8dc742ed8a35eccb7c570f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:29.985996 28472 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Bootstrap starting.
I20260809 16:38:29.986670 28472 tablet_bootstrap.cc:654] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.986737 28473 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Bootstrap starting.
I20260809 16:38:29.987110 28472 log.cc:824] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.987389 28473 tablet_bootstrap.cc:654] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.987380 28475 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Bootstrap starting.
I20260809 16:38:29.987710 28473 log.cc:824] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.987913 28472 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: No bootstrap required, opened a new log
I20260809 16:38:29.987983 28472 ts_tablet_manager.cc:1397] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:29.988147 28475 tablet_bootstrap.cc:654] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:29.988397 28473 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: No bootstrap required, opened a new log
I20260809 16:38:29.988458 28473 ts_tablet_manager.cc:1397] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:38:29.988449 28475 log.cc:824] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:29.989230 28475 tablet_bootstrap.cc:492] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: No bootstrap required, opened a new log
I20260809 16:38:29.989300 28475 ts_tablet_manager.cc:1397] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 16:38:29.990502 28473 raft_consensus.cc:348] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.990469 28472 raft_consensus.cc:348] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.990622 28473 raft_consensus.cc:374] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.990623 28472 raft_consensus.cc:374] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.990655 28472 raft_consensus.cc:729] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 276f6ad754a74046a1e9a171d4160a7b, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.990655 28473 raft_consensus.cc:729] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5acbc8f01e974aecb0a22bfa3ae8ad16, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.990768 28472 consensus_queue.cc:260] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.990787 28473 consensus_queue.cc:260] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.991024 28473 ts_tablet_manager.cc:1428] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:38:29.991025 28472 ts_tablet_manager.cc:1428] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:29.991129 28043 heartbeater.cc:503] Master 127.31.15.254:45637 was elected leader, sending a full tablet report...
I20260809 16:38:29.991137 28447 heartbeater.cc:503] Master 127.31.15.254:45637 was elected leader, sending a full tablet report...
I20260809 16:38:29.991667 28475 raft_consensus.cc:348] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.991816 28475 raft_consensus.cc:374] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:29.991863 28475 raft_consensus.cc:729] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d9c4e9d16b43bca40faab1304fcd50, State: Initialized, Role: FOLLOWER
I20260809 16:38:29.991953 28475 consensus_queue.cc:260] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:29.992187 28475 ts_tablet_manager.cc:1428] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 16:38:29.992226 28246 heartbeater.cc:503] Master 127.31.15.254:45637 was elected leader, sending a full tablet report...
W20260809 16:38:30.086094 28247 tablet.cc:2369] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:30.190598 28482 raft_consensus.cc:482] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:30.190748 28482 raft_consensus.cc:504] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:30.191104 28482 leader_election.cc:288] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8d9c4e9d16b43bca40faab1304fcd50 (127.31.15.194:33943), 5acbc8f01e974aecb0a22bfa3ae8ad16 (127.31.15.193:46051)
I20260809 16:38:30.194926 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.191391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:52699 (local address 127.31.15.194:33943)
0809 16:38:30.191447 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:30.191449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:30.191462 (+ 13us) server_negotiation.cc:407] Connection header received
0809 16:38:30.191500 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:30.191501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:30.191511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:30.191599 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:30.191750 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.192530 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194003 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.194591 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194662 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:30.194672 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:30.194807 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:38:30.194849 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:38:30.194927 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.191519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:39255 (local address 127.31.15.193:46051)
0809 16:38:30.191563 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:30.191565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:30.191649 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:38:30.191686 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:30.191687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:30.191698 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:30.191762 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:30.191856 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.192552 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194000 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.194573 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194662 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:30.194672 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:30.194807 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 16:38:30.194847 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:38:30.194934 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.191318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:33943 (local address 127.31.15.195:52699)
0809 16:38:30.191367 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:30.191379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:30.191426 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:30.191608 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:30.191609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:30.191611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:30.191686 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:30.191688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.192633 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.192634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:30.193904 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:30.193908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.194658 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:30.194736 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:30.194740 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:30.194794 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:38:30.194843 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":45}
I20260809 16:38:30.194939 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.191426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46051 (local address 127.31.15.195:39255)
0809 16:38:30.191583 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:30.191592 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:30.191642 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:30.191765 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:30.191766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:30.191767 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:30.191813 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:30.191814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.192633 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.192634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:30.193906 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:30.193908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.194660 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.194661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:30.194742 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:30.194745 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:30.194794 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 16:38:30.194842 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":61,"thread_start_us":103,"threads_started":1}
I20260809 16:38:30.195349 27966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2263597d8dc742ed8a35eccb7c570f83" candidate_uuid: "276f6ad754a74046a1e9a171d4160a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" is_pre_election: true
I20260809 16:38:30.195523 27966 raft_consensus.cc:2454] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 276f6ad754a74046a1e9a171d4160a7b in term 0.
I20260809 16:38:30.195482 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2263597d8dc742ed8a35eccb7c570f83" candidate_uuid: "276f6ad754a74046a1e9a171d4160a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" is_pre_election: true
I20260809 16:38:30.195612 28172 raft_consensus.cc:2454] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 276f6ad754a74046a1e9a171d4160a7b in term 0.
I20260809 16:38:30.195786 28273 leader_election.cc:302] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276f6ad754a74046a1e9a171d4160a7b, 5acbc8f01e974aecb0a22bfa3ae8ad16; no voters:
I20260809 16:38:30.195914 28482 raft_consensus.cc:2781] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:30.195983 28482 raft_consensus.cc:482] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:30.196012 28482 raft_consensus.cc:3037] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:30.196766 28482 raft_consensus.cc:504] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:30.196894 28482 leader_election.cc:288] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [CANDIDATE]: Term 1 election: Requested vote from peers b8d9c4e9d16b43bca40faab1304fcd50 (127.31.15.194:33943), 5acbc8f01e974aecb0a22bfa3ae8ad16 (127.31.15.193:46051)
I20260809 16:38:30.197036 28172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2263597d8dc742ed8a35eccb7c570f83" candidate_uuid: "276f6ad754a74046a1e9a171d4160a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d9c4e9d16b43bca40faab1304fcd50"
I20260809 16:38:30.197084 27966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2263597d8dc742ed8a35eccb7c570f83" candidate_uuid: "276f6ad754a74046a1e9a171d4160a7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16"
I20260809 16:38:30.197119 28172 raft_consensus.cc:3037] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:30.197160 27966 raft_consensus.cc:3037] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:30.197875 27966 raft_consensus.cc:2454] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276f6ad754a74046a1e9a171d4160a7b in term 1.
I20260809 16:38:30.197885 28172 raft_consensus.cc:2454] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 276f6ad754a74046a1e9a171d4160a7b in term 1.
I20260809 16:38:30.198037 28273 leader_election.cc:302] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 276f6ad754a74046a1e9a171d4160a7b, b8d9c4e9d16b43bca40faab1304fcd50; no voters:
I20260809 16:38:30.198143 28482 raft_consensus.cc:2781] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:30.198285 28482 raft_consensus.cc:686] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 1 LEADER]: Becoming Leader. State: Replica: 276f6ad754a74046a1e9a171d4160a7b, State: Running, Role: LEADER
I20260809 16:38:30.198364 28482 consensus_queue.cc:237] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } }
I20260809 16:38:30.199215 27736 catalog_manager.cc:5495] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 276f6ad754a74046a1e9a171d4160a7b (127.31.15.195). New cstate: current_term: 1 leader_uuid: "276f6ad754a74046a1e9a171d4160a7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "276f6ad754a74046a1e9a171d4160a7b" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 41681 } health_report { overall_health: HEALTHY } } }
I20260809 16:38:30.208966 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.206759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.31.15.195:41681)
0809 16:38:30.206817 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:30.206819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:30.206826 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:38:30.206858 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:30.206859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:30.206875 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:30.206936 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:30.207052 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.207546 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.208264 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:30.208445 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:30.208504 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:30.208651 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:30.208756 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:30.208828 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:30.208866 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:38:30.208883 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 16:38:30.210188 28448 tablet.cc:2369] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:30.210580 28172 raft_consensus.cc:1261] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 1 FOLLOWER]: Refusing update from remote peer 276f6ad754a74046a1e9a171d4160a7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:30.210608 27966 raft_consensus.cc:1261] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 1 FOLLOWER]: Refusing update from remote peer 276f6ad754a74046a1e9a171d4160a7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:30.210944 28489 consensus_queue.cc:1035] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5acbc8f01e974aecb0a22bfa3ae8ad16" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:30.211074 28482 consensus_queue.cc:1035] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d9c4e9d16b43bca40faab1304fcd50" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 33943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 16:38:30.212256 28045 tablet.cc:2369] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:30.216207 27736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:30.216300 27736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:30.217061 27736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aedfa7d8158a499e83f9541485f7f941: Sending DeleteTablet for 3 replicas of tablet 2263597d8dc742ed8a35eccb7c570f83
I20260809 16:38:30.217509 27930 tablet_service.cc:1514] Processing DeleteTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:30 UTC) from {username='slave'} at 127.0.0.1:59608
I20260809 16:38:30.217505 28146 tablet_service.cc:1514] Processing DeleteTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:30 UTC) from {username='slave'} at 127.0.0.1:57046
I20260809 16:38:30.217506 28345 tablet_service.cc:1514] Processing DeleteTablet for tablet 2263597d8dc742ed8a35eccb7c570f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:30 UTC) from {username='slave'} at 127.0.0.1:48830
I20260809 16:38:30.217738 28505 tablet_replica.cc:330] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16: stopping tablet replica
I20260809 16:38:30.217829 28505 raft_consensus.cc:2229] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:30.217937 28505 raft_consensus.cc:2258] T 2263597d8dc742ed8a35eccb7c570f83 P 5acbc8f01e974aecb0a22bfa3ae8ad16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:30.218086 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 27841
I20260809 16:38:30.231527 28506 tablet_replica.cc:330] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: stopping tablet replica
I20260809 16:38:30.231660 28506 raft_consensus.cc:2229] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:38:30.231781 28506 raft_consensus.cc:2258] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:30.232471 28506 ts_tablet_manager.cc:1905] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:30.233775 28508 tablet_replica.cc:330] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: stopping tablet replica
I20260809 16:38:30.233903 28508 raft_consensus.cc:2229] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:30.234031 28508 raft_consensus.cc:2258] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:30.234892 28508 ts_tablet_manager.cc:1905] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:30.236873 28508 ts_tablet_manager.cc:1918] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:30.236954 28508 log.cc:1196] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/2263597d8dc742ed8a35eccb7c570f83
I20260809 16:38:30.237211 28508 ts_tablet_manager.cc:1939] T 2263597d8dc742ed8a35eccb7c570f83 P b8d9c4e9d16b43bca40faab1304fcd50: Deleting consensus metadata
I20260809 16:38:30.237692 27713 catalog_manager.cc:4841] TS b8d9c4e9d16b43bca40faab1304fcd50 (127.31.15.194:33943): tablet 2263597d8dc742ed8a35eccb7c570f83 (table test-table [id=e45174fa31d84f72b0c6bc9480712da9]) successfully deleted
I20260809 16:38:30.237867 28506 ts_tablet_manager.cc:1918] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:30.237950 28506 log.cc:1196] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/2263597d8dc742ed8a35eccb7c570f83
I20260809 16:38:30.238214 28506 ts_tablet_manager.cc:1939] T 2263597d8dc742ed8a35eccb7c570f83 P 276f6ad754a74046a1e9a171d4160a7b: Deleting consensus metadata
I20260809 16:38:30.238652 27713 catalog_manager.cc:4841] TS 276f6ad754a74046a1e9a171d4160a7b (127.31.15.195:41681): tablet 2263597d8dc742ed8a35eccb7c570f83 (table test-table [id=e45174fa31d84f72b0c6bc9480712da9]) successfully deleted
W20260809 16:38:30.241595 27713 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:46051 (error 108)
I20260809 16:38:30.242281 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 28052
I20260809 16:38:30.242415 28460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:30.242276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46051 (local address 127.0.0.1:59614)
0809 16:38:30.242328 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:30.242376 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:38:30.242527 27713 catalog_manager.cc:4584] TS 5acbc8f01e974aecb0a22bfa3ae8ad16 (127.31.15.193:46051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2263597d8dc742ed8a35eccb7c570f83: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46051: connect: Connection refused (error 111)
I20260809 16:38:30.250747 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 28249
I20260809 16:38:30.261339 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 27693
2026-08-09T16:38:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (861 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:30 dist-test-slave-21b9 krb5kdc[28515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:30 dist-test-slave-21b9 krb5kdc[28515](info): set up 2 sockets
Aug 09 16:38:30 dist-test-slave-21b9 krb5kdc[28515](info): commencing operation
krb5kdc: starting...
W20260809 16:38:31.404137 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:31.517472 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36259
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:36259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:31.597906 28534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:31.598104 28534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:31.598124 28534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:31.599210 28534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:31.599246 28534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:31.599259 28534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:31.599289 28534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:31.599300 28534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:31.600703 28534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:36259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:36259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:31.600893 28534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:31.601064 28534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:31.605687 28543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.605712 28540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.605890 28546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:31.606458 28534 server_base.cc:1034] running on GCE node
I20260809 16:38:31.606695 28534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:31.606945 28534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:31.608079 28534 hybrid_clock.cc:648] HybridClock initialized: now 1786293511608066 us; error 29 us; skew 500 ppm
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:31.610442 28534 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:31.610850 28534 webserver.cc:460] Webserver started at http://127.31.15.254:37763/ using document root <none> and password file <none>
I20260809 16:38:31.611042 28534 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:31.611084 28534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:31.611202 28534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:31.612284 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "f8c4397f173e42b4ade364b12d4e7d8d"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "280de56b8e186e321b2e8db3f1678919"
server_key_iv: "83e0ef350a0cc1f4a16fba642ec1f035"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.612618 28534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "f8c4397f173e42b4ade364b12d4e7d8d"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "280de56b8e186e321b2e8db3f1678919"
server_key_iv: "83e0ef350a0cc1f4a16fba642ec1f035"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.613893 28534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:31.614526 28555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.614681 28534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:31.614764 28534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "f8c4397f173e42b4ade364b12d4e7d8d"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "280de56b8e186e321b2e8db3f1678919"
server_key_iv: "83e0ef350a0cc1f4a16fba642ec1f035"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.614816 28534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:31.626442 28534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:31.628408 28534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:31.628515 28534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:31.632277 28534 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:36259
I20260809 16:38:31.632315 28641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:36259 every 8 connection(s)
I20260809 16:38:31.632622 28534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:31.633483 28642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:31.635696 28642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: Bootstrap starting.
I20260809 16:38:31.636224 28642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:31.636453 28642 log.cc:824] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:31.637064 28642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: No bootstrap required, opened a new log
I20260809 16:38:31.638576 28642 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } }
I20260809 16:38:31.638707 28642 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:31.638728 28642 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c4397f173e42b4ade364b12d4e7d8d, State: Initialized, Role: FOLLOWER
I20260809 16:38:31.638837 28642 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } }
I20260809 16:38:31.638912 28642 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:31.638948 28642 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:31.638980 28642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:31.639494 28642 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } }
I20260809 16:38:31.639580 28642 leader_election.cc:302] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8c4397f173e42b4ade364b12d4e7d8d; no voters:
I20260809 16:38:31.639698 28642 leader_election.cc:288] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:31.639737 28646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:31.639866 28646 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [term 1 LEADER]: Becoming Leader. State: Replica: f8c4397f173e42b4ade364b12d4e7d8d, State: Running, Role: LEADER
I20260809 16:38:31.639935 28642 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:31.639953 28646 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } }
I20260809 16:38:31.640367 28650 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8c4397f173e42b4ade364b12d4e7d8d. Latest consensus state: current_term: 1 leader_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } } }
I20260809 16:38:31.640448 28650 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:31.640545 28647 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8c4397f173e42b4ade364b12d4e7d8d" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 36259 } } }
I20260809 16:38:31.640630 28647 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:31.640676 28661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:31.641198 28661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:31.642158 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 28534
I20260809 16:38:31.642280 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:31.642494 31807 external_mini_cluster.cc:1428] Setting key 0227cf41a43244183104a799db4da333
I20260809 16:38:31.642693 28661 catalog_manager.cc:1347] Generated new cluster ID: 57a36e09dfce4932b0feef36a527a7e7
I20260809 16:38:31.642746 28661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:31.651432 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:31.643097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54154 (local address 127.31.15.254:36259)
0809 16:38:31.643246 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:31.643254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:31.643300 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:38:31.643411 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:31.643418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:31.643473 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:31.643588 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:31.644193 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.645115 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.645843 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.646238 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.646330 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:31.648466 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:31.648470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:31.648470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:31.648482 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:31.649956 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.650266 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.650268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.650268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.650361 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.650540 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.650541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.650541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.650760 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:31.650905 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:31.651008 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 16:38:31.651220 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 16:38:31.660190 28661 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:31.660668 28661 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:31.663553 28661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: Generated new TSK 0
I20260809 16:38:31.663720 28661 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:38:31.687638 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:31.768358 28688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:31.768558 28688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:31.768590 28688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:31.769727 28688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:31.769769 28688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:31.769805 28688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:31.771227 28688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:31.771477 28688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:31.771694 28688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:31.776254 28697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.776263 28701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.776263 28694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:31.776527 28688 server_base.cc:1034] running on GCE node
I20260809 16:38:31.776700 28688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:31.776891 28688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:31.778025 28688 hybrid_clock.cc:648] HybridClock initialized: now 1786293511778012 us; error 29 us; skew 500 ppm
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:31.780404 28688 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:38:31.780789 28688 webserver.cc:460] Webserver started at http://127.31.15.193:34389/ using document root <none> and password file <none>
I20260809 16:38:31.780977 28688 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:31.781018 28688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:31.781134 28688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:31.782140 28688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "21c09cdc5b5b4b70831c196e56e52196"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "e1bb9db214e4021a4e14b222eeb9d8da"
server_key_iv: "d79f5a0c76d1e4893a2284c517a3e582"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.782464 28688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "21c09cdc5b5b4b70831c196e56e52196"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "e1bb9db214e4021a4e14b222eeb9d8da"
server_key_iv: "d79f5a0c76d1e4893a2284c517a3e582"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.783828 28688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:31.784478 28715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.784619 28688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:31.784700 28688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "21c09cdc5b5b4b70831c196e56e52196"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "e1bb9db214e4021a4e14b222eeb9d8da"
server_key_iv: "d79f5a0c76d1e4893a2284c517a3e582"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.784756 28688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:31.802657 28688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:31.804418 28688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:31.804546 28688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:31.804760 28688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:31.805042 28688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:31.805074 28688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.805104 28688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:31.805120 28688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.810595 28688 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:46121
I20260809 16:38:31.810658 28911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:46121 every 8 connection(s)
I20260809 16:38:31.810926 28688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:38:31.812377 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 28688
I20260809 16:38:31.812494 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:31.812673 31807 external_mini_cluster.cc:1428] Setting key cb91b7983ece2830643e9808c493f2f0
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:31.820017 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:31.811639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:54721 (local address 127.31.15.254:36259)
0809 16:38:31.811681 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:31.811682 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:31.811869 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:38:31.812369 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:31.812370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:31.812386 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:31.812468 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:31.813496 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.814001 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.814952 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.815157 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.815212 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:31.817390 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:31.817391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:31.817392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:31.817399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:31.818709 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.819071 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.819072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.819072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.819127 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.819374 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.819375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.819375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.819493 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:31.819547 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:31.819837 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 16:38:31.819929 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:38:31.820186 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:31.811614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36259 (local address 127.31.15.193:54721)
0809 16:38:31.811756 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:31.811791 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:31.812274 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:31.812545 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:31.812549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:31.813182 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:31.813414 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:31.813420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.814082 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.814083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:31.814868 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:31.814870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.815227 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.815228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:31.815385 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:31.815837 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:31.815845 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:31.817279 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:31.818777 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:31.818785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:31.818787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:31.819004 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:31.819181 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:31.819182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:31.819182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:31.819315 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:31.819547 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:31.819554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:31.819706 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 16:38:31.819809 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 16:38:31.819971 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260809 16:38:31.820657 28913 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36259
I20260809 16:38:31.820740 28913 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:31.820904 28913 heartbeater.cc:511] Master 127.31.15.254:36259 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
I20260809 16:38:31.821441 28582 ts_manager.cc:194] Registered new tserver with Master: 21c09cdc5b5b4b70831c196e56e52196 (127.31.15.193:46121)
I20260809 16:38:31.822049 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:54721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:38:31.835282 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:31.915704 28921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:31.915900 28921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:31.915922 28921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:31.917037 28921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:31.917071 28921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:31.917104 28921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:31.918576 28921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:31.918762 28921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:31.918946 28921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:31.923732 28932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.923740 28929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:31.923744 28930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:31.924045 28921 server_base.cc:1034] running on GCE node
I20260809 16:38:31.924232 28921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:31.924424 28921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:31.925551 28921 hybrid_clock.cc:648] HybridClock initialized: now 1786293511925537 us; error 29 us; skew 500 ppm
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:31.927910 28921 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:38:31.928305 28921 webserver.cc:460] Webserver started at http://127.31.15.194:34139/ using document root <none> and password file <none>
I20260809 16:38:31.928478 28921 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:31.928517 28921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:31.928614 28921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:31.929656 28921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "eaaf2efd32644a82b5cd320f806c29f3"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "933b3a86fc9e274b3a6e97012051e62f"
server_key_iv: "b2e8f600becf01b20d2e1ce64b5b1539"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.929949 28921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "eaaf2efd32644a82b5cd320f806c29f3"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "933b3a86fc9e274b3a6e97012051e62f"
server_key_iv: "b2e8f600becf01b20d2e1ce64b5b1539"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.931133 28921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:31.931833 28941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.931985 28921 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.932072 28921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "eaaf2efd32644a82b5cd320f806c29f3"
format_stamp: "Formatted at 2026-08-09 16:38:31 on dist-test-slave-21b9"
server_key: "933b3a86fc9e274b3a6e97012051e62f"
server_key_iv: "b2e8f600becf01b20d2e1ce64b5b1539"
server_key_version: "encryptionkey@0"
I20260809 16:38:31.932130 28921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:31.958110 28921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:31.960347 28921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:31.960474 28921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:31.960678 28921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:31.960960 28921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:31.960994 28921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.961015 28921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:31.961030 28921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:31.966373 28921 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:34103
I20260809 16:38:31.966446 29132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:34103 every 8 connection(s)
I20260809 16:38:31.966737 28921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:31.969970 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 28921
I20260809 16:38:31.970103 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:31.970304 31807 external_mini_cluster.cc:1428] Setting key b91110acd6b40d611044bd2b0a7bcc05
Aug 09 16:38:31 dist-test-slave-21b9 krb5kdc[28515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293511, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:31.976089 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:31.967444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:44289 (local address 127.31.15.254:36259)
0809 16:38:31.967512 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:31.967514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:31.967714 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:38:31.968203 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:31.968204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:31.968214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:31.968267 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:31.969199 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.969717 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.970924 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.971193 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.971285 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:31.973430 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:31.973432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:31.973432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:31.973439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:31.974775 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.975151 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.975153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.975153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.975219 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:31.975471 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:31.975472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:31.975472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:31.975614 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:31.975677 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:31.975906 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 16:38:31.975991 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:38:31.976154 29136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:31.967419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36259 (local address 127.31.15.194:44289)
0809 16:38:31.967593 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:31.967630 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:31.968101 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:31.968323 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:31.968326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:31.968884 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:31.969076 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:31.969090 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.969798 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.969799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:31.970842 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:31.970845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:31.971300 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:31.971301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:31.971428 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:31.971920 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:31.971929 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:31.973345 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:31.974852 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:31.974858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:31.974860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:31.975074 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:31.975305 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:31.975306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:31.975307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:31.975409 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:31.975675 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:31.975678 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:31.975789 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 16:38:31.975864 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 16:38:31.975983 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260809 16:38:31.976577 29133 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36259
I20260809 16:38:31.976691 29133 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:31.976886 29133 heartbeater.cc:511] Master 127.31.15.254:36259 requested a full tablet report, sending...
I20260809 16:38:31.977265 28582 ts_manager.cc:194] Registered new tserver with Master: eaaf2efd32644a82b5cd320f806c29f3 (127.31.15.194:34103)
I20260809 16:38:31.977737 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:44289
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:38:31.995240 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:32.078393 29140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:32.078614 29140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:32.078650 29140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:32.079814 29140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:32.079855 29140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:32.079890 29140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:32.081296 29140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:36259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:32.081543 29140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:32.081759 29140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:32.086529 29151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.086618 29154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.086536 29152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.086722 29140 server_base.cc:1034] running on GCE node
I20260809 16:38:32.086926 29140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:32.087144 29140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:32.088295 29140 hybrid_clock.cc:648] HybridClock initialized: now 1786293512088276 us; error 29 us; skew 500 ppm
Aug 09 16:38:32 dist-test-slave-21b9 krb5kdc[28515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293512, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:32.090829 29140 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:38:32.091248 29140 webserver.cc:460] Webserver started at http://127.31.15.195:42955/ using document root <none> and password file <none>
I20260809 16:38:32.091475 29140 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:32.091522 29140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:32.091648 29140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:32.092664 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "dfabd5cbd180429290b8d62cb2ff2510"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "1f1dcd8ffe35cec0b1cb78b7c29e95ce"
server_key_iv: "9507109d7c66a11fdc03e395672e98a7"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.092994 29140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "dfabd5cbd180429290b8d62cb2ff2510"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "1f1dcd8ffe35cec0b1cb78b7c29e95ce"
server_key_iv: "9507109d7c66a11fdc03e395672e98a7"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.094313 29140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:32.095017 29163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.095177 29140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:32.095261 29140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "dfabd5cbd180429290b8d62cb2ff2510"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "1f1dcd8ffe35cec0b1cb78b7c29e95ce"
server_key_iv: "9507109d7c66a11fdc03e395672e98a7"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.095348 29140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:32.114395 29140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:32.116786 29140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:32.116951 29140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:32.117221 29140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:32.117540 29140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:32.117575 29140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.117604 29140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:32.117627 29140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.123241 29140 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:40119
I20260809 16:38:32.123337 29354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:40119 every 8 connection(s)
I20260809 16:38:32.123633 29140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:38:32 dist-test-slave-21b9 krb5kdc[28515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293512, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:32.129892 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 29140
I20260809 16:38:32.130008 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:32.130227 31807 external_mini_cluster.cc:1428] Setting key 3537e7a5d41fe4ea9be1529de8b4bfe4
I20260809 16:38:32.132450 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.124318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:52371 (local address 127.31.15.254:36259)
0809 16:38:32.124396 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.124398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.124574 (+ 176us) server_negotiation.cc:407] Connection header received
0809 16:38:32.125045 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.125046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.125056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.125107 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.126008 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.126504 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.127474 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.127654 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.127703 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.129787 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.129789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.129789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:32.129797 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.131398 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:32.131755 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:32.131755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:32.131756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:32.131803 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:32.131983 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:32.131983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:32.131984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:32.132077 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.132120 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.132326 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.132387 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:38:32.132658 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.124294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:36259 (local address 127.31.15.195:52371)
0809 16:38:32.124445 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.124496 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.124967 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.125160 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.125163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.125726 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:32.125921 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.125926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.126567 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.126567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.127410 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.127412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.127710 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.127711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.127804 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.128279 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:32.128288 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:32.129685 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:32.131466 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:32.131473 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:32.131474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:32.131693 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:32.131845 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:32.131846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:32.131846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:32.131941 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:32.132117 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:32.132120 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:32.132230 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.132309 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.132452 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260809 16:38:32.133040 29355 heartbeater.cc:347] Connected to a master server at 127.31.15.254:36259
I20260809 16:38:32.133128 29355 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:32.133289 29355 heartbeater.cc:511] Master 127.31.15.254:36259 requested a full tablet report, sending...
I20260809 16:38:32.133639 28582 ts_manager.cc:194] Registered new tserver with Master: dfabd5cbd180429290b8d62cb2ff2510 (127.31.15.195:40119)
I20260809 16:38:32.134008 28582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:52371
I20260809 16:38:32.141963 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:32.148476 28681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.143238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54166 (local address 127.31.15.254:36259)
0809 16:38:32.143314 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.143316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.143404 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:38:32.143467 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.143468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.143476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.143520 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.144022 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.144482 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.145123 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.145288 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.145327 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.146447 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.146448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.146448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:32.146454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.147574 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:32.147913 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:32.147913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:32.147914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:32.147959 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:32.148085 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:32.148085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:32.148086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:32.148192 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.148265 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.148347 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.148390 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 16:38:32.149984 28582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:32.151558 28582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:32.157503 29370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.153431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34103 (local address 127.0.0.1:40990)
0809 16:38:32.153594 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.153634 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.153699 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.154073 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.154076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.154079 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.154168 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.154170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155525 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.155526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.156571 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.156573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157208 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.157286 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157300 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.157365 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.157417 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260809 16:38:32.157603 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.153444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46121 (local address 127.0.0.1:50410)
0809 16:38:32.153610 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.153635 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.153699 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.154087 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.154088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.154090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.154147 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.154149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155558 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.155559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.156423 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.156426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157322 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.157395 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157399 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.157440 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.157482 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260809 16:38:32.157752 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.153559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.31.15.194:34103)
0809 16:38:32.153907 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.153912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.153919 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:38:32.153949 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.153951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.153977 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.154057 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.154251 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155425 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.156631 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157131 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157195 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157256 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.157623 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.157676 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":88,"threads_started":1}
I20260809 16:38:32.157781 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.153745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.31.15.193:46121)
0809 16:38:32.153908 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.153913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.153923 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:32.153964 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.153966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.153981 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.154224 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.154307 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155437 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.156501 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157215 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157555 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157632 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.157658 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.157701 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260809 16:38:32.157809 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.153632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40119 (local address 127.0.0.1:52770)
0809 16:38:32.153783 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.153791 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.153819 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.154352 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.154353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.154355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.154405 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.154407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155297 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.155298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.156514 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.156516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157338 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.157395 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157398 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.157445 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.157485 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":1,"thread_start_us":41,"threads_started":1}
I20260809 16:38:32.158393 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.154014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.31.15.195:40119)
0809 16:38:32.154198 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.154201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.154211 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:32.154245 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.154246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.154261 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.154486 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.154540 (+ 54us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.155192 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.156598 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.157276 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.157339 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.157398 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.157459 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.157500 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260809 16:38:32.158510 29022 tablet_service.cc:1467] Processing CreateTablet for tablet f1351118d53b4758be50087495741df7 (DEFAULT_TABLE table=test-table [id=893fda3ffb83473da15a7ed53c4c14d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.158560 28803 tablet_service.cc:1467] Processing CreateTablet for tablet f1351118d53b4758be50087495741df7 (DEFAULT_TABLE table=test-table [id=893fda3ffb83473da15a7ed53c4c14d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.158902 29022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1351118d53b4758be50087495741df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.158910 28803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1351118d53b4758be50087495741df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.159062 29247 tablet_service.cc:1467] Processing CreateTablet for tablet f1351118d53b4758be50087495741df7 (DEFAULT_TABLE table=test-table [id=893fda3ffb83473da15a7ed53c4c14d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.159384 29247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1351118d53b4758be50087495741df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.161586 29378 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Bootstrap starting.
I20260809 16:38:32.162257 29378 tablet_bootstrap.cc:654] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.162547 29380 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Bootstrap starting.
I20260809 16:38:32.162848 29378 log.cc:824] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.163100 29380 tablet_bootstrap.cc:654] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.163443 29379 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: Bootstrap starting.
I20260809 16:38:32.163542 29378 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: No bootstrap required, opened a new log
I20260809 16:38:32.163614 29378 ts_tablet_manager.cc:1397] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.163628 29380 log.cc:824] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.164111 29379 tablet_bootstrap.cc:654] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.164418 29380 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: No bootstrap required, opened a new log
I20260809 16:38:32.164494 29380 ts_tablet_manager.cc:1397] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.164644 29379 log.cc:824] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.165354 29378 raft_consensus.cc:348] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.165496 29378 raft_consensus.cc:374] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.165518 29379 tablet_bootstrap.cc:492] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: No bootstrap required, opened a new log
I20260809 16:38:32.165529 29378 raft_consensus.cc:729] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaaf2efd32644a82b5cd320f806c29f3, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.165591 29379 ts_tablet_manager.cc:1397] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.165693 29378 consensus_queue.cc:260] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.165982 29378 ts_tablet_manager.cc:1428] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 16:38:32.166114 29133 heartbeater.cc:503] Master 127.31.15.254:36259 was elected leader, sending a full tablet report...
I20260809 16:38:32.166290 29380 raft_consensus.cc:348] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.166564 29380 raft_consensus.cc:374] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.166611 29380 raft_consensus.cc:729] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfabd5cbd180429290b8d62cb2ff2510, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.166730 29380 consensus_queue.cc:260] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.166998 29380 ts_tablet_manager.cc:1428] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:38:32.167130 29355 heartbeater.cc:503] Master 127.31.15.254:36259 was elected leader, sending a full tablet report...
I20260809 16:38:32.167579 29379 raft_consensus.cc:348] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.167707 29379 raft_consensus.cc:374] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.167738 29379 raft_consensus.cc:729] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21c09cdc5b5b4b70831c196e56e52196, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.167840 29379 consensus_queue.cc:260] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.168088 29379 ts_tablet_manager.cc:1428] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:38:32.168119 28913 heartbeater.cc:503] Master 127.31.15.254:36259 was elected leader, sending a full tablet report...
W20260809 16:38:32.217295 29135 tablet.cc:2369] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:32.225328 29387 raft_consensus.cc:482] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:32.225451 29387 raft_consensus.cc:504] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.225854 29387 leader_election.cc:288] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaaf2efd32644a82b5cd320f806c29f3 (127.31.15.194:34103), dfabd5cbd180429290b8d62cb2ff2510 (127.31.15.195:40119)
I20260809 16:38:32.229610 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.226099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:39939 (local address 127.31.15.194:34103)
0809 16:38:32.226148 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.226150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.226162 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:38:32.226266 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.226267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.226277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.226346 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.226529 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.227302 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.228664 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.229250 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.229324 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.229333 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.229505 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.229538 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:38:32.229609 29376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.226296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:38085 (local address 127.31.15.195:40119)
0809 16:38:32.226353 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.226355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.226445 (+ 90us) server_negotiation.cc:407] Connection header received
0809 16:38:32.226491 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.226492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.226500 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.226558 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.226745 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.227437 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.228660 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.229250 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.229324 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.229333 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.229500 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.229537 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:38:32.229662 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.226052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:34103 (local address 127.31.15.193:39939)
0809 16:38:32.226109 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.226122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.226170 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.226363 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.226365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.226367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.226437 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.226439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.227379 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.227380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.228576 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.228579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.229325 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.229325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.229430 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.229436 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.229488 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.229558 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":7}
I20260809 16:38:32.229668 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.226177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:40119 (local address 127.31.15.193:38085)
0809 16:38:32.226378 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.226387 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.226443 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.226561 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.226562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.226563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.226634 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.226635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.227504 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.227505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.228586 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.228588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.229325 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.229325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.229430 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.229437 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.229488 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.229557 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":30,"thread_start_us":117,"threads_started":1}
I20260809 16:38:32.230100 29054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1351118d53b4758be50087495741df7" candidate_uuid: "21c09cdc5b5b4b70831c196e56e52196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaaf2efd32644a82b5cd320f806c29f3" is_pre_election: true
I20260809 16:38:32.230258 29054 raft_consensus.cc:2454] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c09cdc5b5b4b70831c196e56e52196 in term 0.
I20260809 16:38:32.230487 28717 leader_election.cc:302] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c09cdc5b5b4b70831c196e56e52196, eaaf2efd32644a82b5cd320f806c29f3; no voters:
I20260809 16:38:32.230620 29281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1351118d53b4758be50087495741df7" candidate_uuid: "21c09cdc5b5b4b70831c196e56e52196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfabd5cbd180429290b8d62cb2ff2510" is_pre_election: true
I20260809 16:38:32.230670 29387 raft_consensus.cc:2781] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:32.230716 29387 raft_consensus.cc:482] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:32.230746 29387 raft_consensus.cc:3037] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:32.230763 29281 raft_consensus.cc:2454] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21c09cdc5b5b4b70831c196e56e52196 in term 0.
I20260809 16:38:32.231576 29387 raft_consensus.cc:504] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.231693 29387 leader_election.cc:288] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [CANDIDATE]: Term 1 election: Requested vote from peers eaaf2efd32644a82b5cd320f806c29f3 (127.31.15.194:34103), dfabd5cbd180429290b8d62cb2ff2510 (127.31.15.195:40119)
I20260809 16:38:32.231879 29281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1351118d53b4758be50087495741df7" candidate_uuid: "21c09cdc5b5b4b70831c196e56e52196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfabd5cbd180429290b8d62cb2ff2510"
I20260809 16:38:32.231864 29054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1351118d53b4758be50087495741df7" candidate_uuid: "21c09cdc5b5b4b70831c196e56e52196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaaf2efd32644a82b5cd320f806c29f3"
I20260809 16:38:32.231947 29281 raft_consensus.cc:3037] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:32.231948 29054 raft_consensus.cc:3037] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:32.232843 29054 raft_consensus.cc:2454] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c09cdc5b5b4b70831c196e56e52196 in term 1.
I20260809 16:38:32.232848 29281 raft_consensus.cc:2454] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21c09cdc5b5b4b70831c196e56e52196 in term 1.
I20260809 16:38:32.233047 28717 leader_election.cc:302] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21c09cdc5b5b4b70831c196e56e52196, dfabd5cbd180429290b8d62cb2ff2510; no voters:
I20260809 16:38:32.233132 29387 raft_consensus.cc:2781] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:32.233274 29387 raft_consensus.cc:686] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [term 1 LEADER]: Becoming Leader. State: Replica: 21c09cdc5b5b4b70831c196e56e52196, State: Running, Role: LEADER
I20260809 16:38:32.233389 29387 consensus_queue.cc:237] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } }
I20260809 16:38:32.234243 28582 catalog_manager.cc:5495] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21c09cdc5b5b4b70831c196e56e52196 (127.31.15.193). New cstate: current_term: 1 leader_uuid: "21c09cdc5b5b4b70831c196e56e52196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21c09cdc5b5b4b70831c196e56e52196" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 46121 } health_report { overall_health: HEALTHY } } }
I20260809 16:38:32.249079 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.246954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50416 (local address 127.31.15.193:46121)
0809 16:38:32.246999 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.247001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.247011 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:32.247066 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.247067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.247083 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.247145 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:32.247252 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.247734 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.248469 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.248654 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.248705 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.248807 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:32.248891 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:32.248954 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.248989 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.248997 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:38:32.250835 29054 raft_consensus.cc:1261] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 1 FOLLOWER]: Refusing update from remote peer 21c09cdc5b5b4b70831c196e56e52196: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:32.250834 29281 raft_consensus.cc:1261] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 1 FOLLOWER]: Refusing update from remote peer 21c09cdc5b5b4b70831c196e56e52196: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:32.251170 29387 consensus_queue.cc:1035] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfabd5cbd180429290b8d62cb2ff2510" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 40119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:32.251325 29390 consensus_queue.cc:1035] T f1351118d53b4758be50087495741df7 P 21c09cdc5b5b4b70831c196e56e52196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaaf2efd32644a82b5cd320f806c29f3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 34103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:32.253739 29395 mvcc.cc:205] Tried to move back new op lower bound from 7316658226176802816 to 7316658226108575744. Current Snapshot: MvccSnapshot[applied={T|T < 7316658226176802816}]
I20260809 16:38:32.254510 29397 mvcc.cc:205] Tried to move back new op lower bound from 7316658226176802816 to 7316658226108575744. Current Snapshot: MvccSnapshot[applied={T|T < 7316658226176802816}]
I20260809 16:38:32.255882 28582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:32.255968 28582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:32.256672 28582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8c4397f173e42b4ade364b12d4e7d8d: Sending DeleteTablet for 3 replicas of tablet f1351118d53b4758be50087495741df7
I20260809 16:38:32.257069 29022 tablet_service.cc:1514] Processing DeleteTablet for tablet f1351118d53b4758be50087495741df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:32 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:40990
I20260809 16:38:32.257081 29247 tablet_service.cc:1514] Processing DeleteTablet for tablet f1351118d53b4758be50087495741df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:32 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:52770
I20260809 16:38:32.257114 28803 tablet_service.cc:1514] Processing DeleteTablet for tablet f1351118d53b4758be50087495741df7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:32 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:50410
I20260809 16:38:32.257340 29409 tablet_replica.cc:330] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: stopping tablet replica
I20260809 16:38:32.257385 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 28688
I20260809 16:38:32.257436 29409 raft_consensus.cc:2229] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:32.257525 29409 raft_consensus.cc:2258] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:32.258127 29410 tablet_replica.cc:330] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: stopping tablet replica
I20260809 16:38:32.258213 29410 raft_consensus.cc:2229] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:32.258287 29410 raft_consensus.cc:2258] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:32.267146 29409 ts_tablet_manager.cc:1905] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:32.268502 29410 ts_tablet_manager.cc:1905] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:32.268757 29409 ts_tablet_manager.cc:1918] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:32.268834 29409 log.cc:1196] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/f1351118d53b4758be50087495741df7
I20260809 16:38:32.269098 29409 ts_tablet_manager.cc:1939] T f1351118d53b4758be50087495741df7 P eaaf2efd32644a82b5cd320f806c29f3: Deleting consensus metadata
I20260809 16:38:32.269615 28556 catalog_manager.cc:4841] TS eaaf2efd32644a82b5cd320f806c29f3 (127.31.15.194:34103): tablet f1351118d53b4758be50087495741df7 (table test-table [id=893fda3ffb83473da15a7ed53c4c14d4]) successfully deleted
I20260809 16:38:32.270051 29410 ts_tablet_manager.cc:1918] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:32.270112 29410 log.cc:1196] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/f1351118d53b4758be50087495741df7
I20260809 16:38:32.270334 29410 ts_tablet_manager.cc:1939] T f1351118d53b4758be50087495741df7 P dfabd5cbd180429290b8d62cb2ff2510: Deleting consensus metadata
I20260809 16:38:32.270776 28556 catalog_manager.cc:4841] TS dfabd5cbd180429290b8d62cb2ff2510 (127.31.15.195:40119): tablet f1351118d53b4758be50087495741df7 (table test-table [id=893fda3ffb83473da15a7ed53c4c14d4]) successfully deleted
W20260809 16:38:32.272805 28558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:46121 (error 108)
I20260809 16:38:32.272987 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 28921
I20260809 16:38:32.276297 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.276157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:46121 (local address 127.0.0.1:50424)
0809 16:38:32.276214 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.276263 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 16:38:32.276489 28558 catalog_manager.cc:4584] TS 21c09cdc5b5b4b70831c196e56e52196 (127.31.15.193:46121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f1351118d53b4758be50087495741df7: Network error: Client connection negotiation failed: client connection to 127.31.15.193:46121: connect: Connection refused (error 111)
I20260809 16:38:32.285030 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 29140
I20260809 16:38:32.294284 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 28534
2026-08-09T16:38:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2034 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T16:38:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:32Z Disabled control of system clock
I20260809 16:38:32.326812 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46057
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:32.408439 29418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:32.408686 29418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:32.408718 29418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:32.409909 29418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:32.409950 29418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:32.409966 29418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:32.409982 29418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:32.409997 29418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:32.411371 29418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46057
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:32.411633 29418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:32.411919 29418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:32.416718 29426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.416718 29428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.416754 29418 server_base.cc:1034] running on GCE node
W20260809 16:38:32.416715 29425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.417160 29418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:32.417464 29418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:32.418619 29418 hybrid_clock.cc:648] HybridClock initialized: now 1786293512418599 us; error 34 us; skew 500 ppm
I20260809 16:38:32.422577 29418 webserver.cc:460] Webserver started at http://127.31.15.254:38097/ using document root <none> and password file <none>
I20260809 16:38:32.422821 29418 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:32.422869 29418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:32.423004 29418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:32.424129 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "77c0fe2b870e4c749cd08cfab6699f2c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "8b5e1d23b12248cbd86d3ab228124a60"
server_key_iv: "1b2bec6871a3adf396f5fcd2e068c7b6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.424474 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "77c0fe2b870e4c749cd08cfab6699f2c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "8b5e1d23b12248cbd86d3ab228124a60"
server_key_iv: "1b2bec6871a3adf396f5fcd2e068c7b6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.425730 29418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:38:32.426440 29435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.426616 29418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:32.426705 29418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "77c0fe2b870e4c749cd08cfab6699f2c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "8b5e1d23b12248cbd86d3ab228124a60"
server_key_iv: "1b2bec6871a3adf396f5fcd2e068c7b6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.426771 29418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:32.450307 29418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:32.450666 29418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:32.450804 29418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:32.454713 29418 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:46057
I20260809 16:38:32.454752 29524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:46057 every 8 connection(s)
I20260809 16:38:32.455084 29418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:32.456086 29526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.458379 29526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: Bootstrap starting.
I20260809 16:38:32.458958 29526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.459245 29526 log.cc:824] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.459942 29526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: No bootstrap required, opened a new log
I20260809 16:38:32.461354 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 29418
I20260809 16:38:32.461468 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:32.461654 31807 external_mini_cluster.cc:1428] Setting key a17437099b0862e1f24710980238604a
I20260809 16:38:32.461597 29526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } }
I20260809 16:38:32.461747 29526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.461790 29526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77c0fe2b870e4c749cd08cfab6699f2c, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.461926 29526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } }
I20260809 16:38:32.462001 29526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:32.462044 29526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:32.462082 29526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:32.462657 29526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } }
I20260809 16:38:32.462762 29526 leader_election.cc:302] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77c0fe2b870e4c749cd08cfab6699f2c; no voters:
I20260809 16:38:32.462927 29526 leader_election.cc:288] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:32.463016 29532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:32.463196 29532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [term 1 LEADER]: Becoming Leader. State: Replica: 77c0fe2b870e4c749cd08cfab6699f2c, State: Running, Role: LEADER
I20260809 16:38:32.463203 29526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:32.463376 29532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } }
I20260809 16:38:32.463774 29533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } } }
I20260809 16:38:32.463793 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77c0fe2b870e4c749cd08cfab6699f2c. Latest consensus state: current_term: 1 leader_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77c0fe2b870e4c749cd08cfab6699f2c" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46057 } } }
I20260809 16:38:32.463851 29533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:32.463876 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:32.464056 29542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:32.464509 29542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:32.466116 29542 catalog_manager.cc:1347] Generated new cluster ID: 1a083c5638b64e0ba2b05b665ad2fb95
I20260809 16:38:32.466171 29542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:32.467772 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.462139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.31.15.254:46057)
0809 16:38:32.462366 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.462373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.462397 (+ 24us) server_negotiation.cc:407] Connection header received
0809 16:38:32.462548 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.462554 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.462615 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.462732 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.462990 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.464702 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.465826 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.466339 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.466452 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.467100 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.467104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.467104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:32.467297 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.467340 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:32.467354 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.467413 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.467484 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.467570 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":95,"threads_started":1}
I20260809 16:38:32.487571 29542 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:32.488137 29542 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:32.491143 29542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: Generated new TSK 0
I20260809 16:38:32.491333 29542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:32.494653 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--builtin_ntp_servers=127.31.15.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:32.579082 29556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:32.579335 29556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:32.579366 29556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:32.580497 29556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:32.580539 29556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:32.580572 29556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:32.581902 29556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:32.582156 29556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:32.582453 29556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:32.588089 29562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.588176 29556 server_base.cc:1034] running on GCE node
W20260809 16:38:32.588131 29566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.588367 29561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.588569 29556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:32.588842 29556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:32.590014 29556 hybrid_clock.cc:648] HybridClock initialized: now 1786293512589995 us; error 32 us; skew 500 ppm
I20260809 16:38:32.591428 29556 webserver.cc:460] Webserver started at http://127.31.15.193:42425/ using document root <none> and password file <none>
I20260809 16:38:32.591691 29556 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:32.591749 29556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:32.591893 29556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:32.593262 29556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "375a16bd1f5b416bb20eaf76b3a9a52c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "5abad4fee6c6af5d889a0f173ef9756b"
server_key_iv: "80d4f6f5c1d55879e31c0027dcee71e6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.593634 29556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "375a16bd1f5b416bb20eaf76b3a9a52c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "5abad4fee6c6af5d889a0f173ef9756b"
server_key_iv: "80d4f6f5c1d55879e31c0027dcee71e6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.594820 29556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:32.595504 29579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.595660 29556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:32.595746 29556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "375a16bd1f5b416bb20eaf76b3a9a52c"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "5abad4fee6c6af5d889a0f173ef9756b"
server_key_iv: "80d4f6f5c1d55879e31c0027dcee71e6"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.595818 29556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:32.616870 29556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:32.617194 29556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:32.617326 29556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:32.617556 29556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:32.617897 29556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:32.617959 29556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.617992 29556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:32.618017 29556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.623858 29556 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:32823
I20260809 16:38:32.623916 29767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:32823 every 8 connection(s)
I20260809 16:38:32.624228 29556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:38:32.629169 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.624927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:47783 (local address 127.31.15.254:46057)
0809 16:38:32.624982 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.624985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.625178 (+ 193us) server_negotiation.cc:407] Connection header received
0809 16:38:32.625684 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.625685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.625700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.625752 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.626433 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.626929 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.627910 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.628114 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.628170 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.628872 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.628873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.628874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:32.628879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.628893 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:32.628897 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.628941 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.629043 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.629077 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:38:32.629406 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.624895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46057 (local address 127.31.15.193:47783)
0809 16:38:32.625063 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.625097 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.625590 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.625819 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.625823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.625830 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:32.626336 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.626341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.627015 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.627016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.627826 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.627828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.628183 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.628184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.628305 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.628749 (+ 444us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:32.628750 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:32.628777 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:32.628799 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:32.628953 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:32.628958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:32.628961 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.629043 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.629155 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260809 16:38:32.629920 29768 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46057
I20260809 16:38:32.630007 29768 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:32.630162 29768 heartbeater.cc:511] Master 127.31.15.254:46057 requested a full tablet report, sending...
I20260809 16:38:32.630663 29464 ts_manager.cc:194] Registered new tserver with Master: 375a16bd1f5b416bb20eaf76b3a9a52c (127.31.15.193:32823)
I20260809 16:38:32.631256 29464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.193:47783
I20260809 16:38:32.632547 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 29556
I20260809 16:38:32.632637 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:32.632792 31807 external_mini_cluster.cc:1428] Setting key 7090fed4ccec8577a2b0253d14d35f41
I20260809 16:38:32.633554 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--builtin_ntp_servers=127.31.15.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:32.713527 29773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:32.713737 29773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:32.713768 29773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:32.714875 29773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:32.714915 29773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:32.714946 29773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:32.716292 29773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:32.716531 29773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:32.716811 29773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:32.721470 29779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.721515 29782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.721647 29773 server_base.cc:1034] running on GCE node
W20260809 16:38:32.721688 29780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.721882 29773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:32.722103 29773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:32.723237 29773 hybrid_clock.cc:648] HybridClock initialized: now 1786293512723228 us; error 26 us; skew 500 ppm
I20260809 16:38:32.724390 29773 webserver.cc:460] Webserver started at http://127.31.15.194:40997/ using document root <none> and password file <none>
I20260809 16:38:32.724586 29773 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:32.724642 29773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:32.724743 29773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:32.725725 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "94c8f252d06b4fd5a080be493cdaacb3"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "0d54f9e3754d80315076985c642ef1bd"
server_key_iv: "bf5df1570634622d5dce3685bdd2aded"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.726011 29773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "94c8f252d06b4fd5a080be493cdaacb3"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "0d54f9e3754d80315076985c642ef1bd"
server_key_iv: "bf5df1570634622d5dce3685bdd2aded"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.727173 29773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:32.727890 29797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.728037 29773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:32.728124 29773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "94c8f252d06b4fd5a080be493cdaacb3"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "0d54f9e3754d80315076985c642ef1bd"
server_key_iv: "bf5df1570634622d5dce3685bdd2aded"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.728197 29773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:32.746874 29773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:32.747156 29773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:32.747308 29773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:32.747516 29773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:32.747826 29773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:32.747860 29773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.747888 29773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:32.747910 29773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.753357 29773 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:38173
I20260809 16:38:32.753417 29977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:38173 every 8 connection(s)
I20260809 16:38:32.753710 29773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:32.758253 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.754387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:52681 (local address 127.31.15.254:46057)
0809 16:38:32.754430 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.754432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.754654 (+ 222us) server_negotiation.cc:407] Connection header received
0809 16:38:32.755139 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.755140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.755148 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.755184 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.755840 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.756327 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.757207 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.757385 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.757429 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.758066 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.758067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.758067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:32.758071 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.758084 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:32.758088 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.758124 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.758175 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.758198 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:38:32.758436 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.754362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46057 (local address 127.31.15.194:52681)
0809 16:38:32.754541 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.754581 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.755035 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.755241 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.755244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.755252 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:32.755770 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.755775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.756387 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.756387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.757149 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.757151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.757436 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.757437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.757535 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.757950 (+ 415us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:32.757951 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:32.757977 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:32.757999 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:32.758125 (+ 126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:32.758127 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:32.758129 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.758171 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.758243 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260809 16:38:32.758665 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 29773
I20260809 16:38:32.758765 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:32.758965 31807 external_mini_cluster.cc:1428] Setting key 277ed3c95f67aa1b7a5cb2764e04db97
I20260809 16:38:32.758993 29978 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46057
I20260809 16:38:32.759095 29978 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:32.759325 29978 heartbeater.cc:511] Master 127.31.15.254:46057 requested a full tablet report, sending...
I20260809 16:38:32.759680 29464 ts_manager.cc:194] Registered new tserver with Master: 94c8f252d06b4fd5a080be493cdaacb3 (127.31.15.194:38173)
I20260809 16:38:32.759898 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--builtin_ntp_servers=127.31.15.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:38:32.760123 29464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.194:52681
W20260809 16:38:32.841174 29984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:32.841367 29984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:32.841387 29984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:32.842433 29984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:32.842466 29984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:32.842489 29984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:32.843799 29984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:46057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:32.843989 29984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:32.844238 29984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:32.848872 30001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.848898 29997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:32.848886 29998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:32.849059 29984 server_base.cc:1034] running on GCE node
I20260809 16:38:32.849269 29984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:32.849479 29984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:32.850623 29984 hybrid_clock.cc:648] HybridClock initialized: now 1786293512850612 us; error 24 us; skew 500 ppm
I20260809 16:38:32.851889 29984 webserver.cc:460] Webserver started at http://127.31.15.195:45049/ using document root <none> and password file <none>
I20260809 16:38:32.852134 29984 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:32.852185 29984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:32.852314 29984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:32.853358 29984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "aea725ec679344bba46c04eb0f96ed1f"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "39d764f71a0d03108d471cdd7b0e993d"
server_key_iv: "cd8b13d500ec868357f41cfaacd67afc"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.853679 29984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "aea725ec679344bba46c04eb0f96ed1f"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "39d764f71a0d03108d471cdd7b0e993d"
server_key_iv: "cd8b13d500ec868357f41cfaacd67afc"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.854868 29984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:32.855545 30007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.855677 29984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:32.855759 29984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "aea725ec679344bba46c04eb0f96ed1f"
format_stamp: "Formatted at 2026-08-09 16:38:32 on dist-test-slave-21b9"
server_key: "39d764f71a0d03108d471cdd7b0e993d"
server_key_iv: "cd8b13d500ec868357f41cfaacd67afc"
server_key_version: "encryptionkey@0"
I20260809 16:38:32.855819 29984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:32.865118 29984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:32.865387 29984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:32.865502 29984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:32.865687 29984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:32.865986 29984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:32.866019 29984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.866056 29984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:32.866081 29984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:32.871372 29984 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:39539
I20260809 16:38:32.871425 30197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:39539 every 8 connection(s)
I20260809 16:38:32.871716 29984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:38:32.874985 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 29984
I20260809 16:38:32.875077 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:32.875262 31807 external_mini_cluster.cc:1428] Setting key 13fd4edd3027293aa76d36f75124b317
I20260809 16:38:32.876698 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.872344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:35161 (local address 127.31.15.254:46057)
0809 16:38:32.872436 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.872438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.872629 (+ 191us) server_negotiation.cc:407] Connection header received
0809 16:38:32.873122 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.873123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.873133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.873184 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.873843 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.874341 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.875342 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.875627 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.875704 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.876506 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.876507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.876507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:32.876511 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.876526 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:32.876530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.876569 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.876616 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.876641 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:38:32.876886 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.872332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46057 (local address 127.31.15.195:35161)
0809 16:38:32.872507 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.872540 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.873034 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.873246 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.873249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.873258 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:32.873749 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.873753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.874408 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.874409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.875230 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.875234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.875718 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.875719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.875878 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.876386 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:32.876386 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:32.876415 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:32.876437 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:32.876571 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:32.876573 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:32.876575 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.876617 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.876692 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260809 16:38:32.877279 30198 heartbeater.cc:347] Connected to a master server at 127.31.15.254:46057
I20260809 16:38:32.877364 30198 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:32.877512 30198 heartbeater.cc:511] Master 127.31.15.254:46057 requested a full tablet report, sending...
I20260809 16:38:32.877825 29464 ts_manager.cc:194] Registered new tserver with Master: aea725ec679344bba46c04eb0f96ed1f (127.31.15.195:39539)
I20260809 16:38:32.878144 29464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.195:35161
I20260809 16:38:32.886896 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:32.890935 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.888218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.31.15.254:46057)
0809 16:38:32.888271 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.888273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.888366 (+ 93us) server_negotiation.cc:407] Connection header received
0809 16:38:32.888441 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.888442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.888452 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.888496 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:32.888653 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.889104 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.889814 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.889993 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.890044 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.890677 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:32.890678 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:32.890679 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:32.890683 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:32.890696 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:32.890700 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:32.890773 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.890845 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.890855 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:38:32.892382 29464 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:32.893833 29464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:32.899776 30218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.895750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:38173 (local address 127.0.0.1:52984)
0809 16:38:32.895906 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.895940 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.896001 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.896515 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.896516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.896518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.896547 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.896548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897487 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.897488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.898665 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.898667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899488 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.899489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.899563 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.899575 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.899646 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.899695 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260809 16:38:32.899780 30221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.896197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52984 (local address 127.31.15.194:38173)
0809 16:38:32.896352 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.896357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.896368 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:38:32.896411 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.896413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.896429 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.896515 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.896601 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897414 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.898758 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899410 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.899486 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.899592 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.899663 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.899700 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260809 16:38:32.899957 30214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.895762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:32823 (local address 127.0.0.1:35048)
0809 16:38:32.896094 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.896102 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.896137 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.896409 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.896410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.896412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.896455 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.896457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897850 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.897851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.898713 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.898716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899739 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.899740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.899797 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.899802 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.899846 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.899883 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260809 16:38:32.900063 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.895750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39539 (local address 127.0.0.1:48070)
0809 16:38:32.895903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.895940 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.896001 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.896372 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.896376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.896378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.896441 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.896444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897819 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.897820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.898899 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.898901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899880 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.899881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.899933 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.899936 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.899963 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.899994 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":30,"thread_start_us":50,"threads_started":1}
I20260809 16:38:32.900179 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.896019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.31.15.193:32823)
0809 16:38:32.896260 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.896264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.896274 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:32.896308 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.896310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.896323 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.896523 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.896598 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897737 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.898799 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899538 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.899614 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.899704 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.900059 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.900098 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260809 16:38:32.900442 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.896025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.31.15.195:39539)
0809 16:38:32.896179 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.896183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.896194 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:38:32.896238 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.896240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.896254 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.896357 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.896627 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.897737 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.898972 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.899793 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.900219 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.900313 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.900336 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.900370 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260809 16:38:32.900760 29871 tablet_service.cc:1467] Processing CreateTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 (DEFAULT_TABLE table=test-table [id=d367aed0b5ae4f18a3173d89f7872ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.900759 29663 tablet_service.cc:1467] Processing CreateTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 (DEFAULT_TABLE table=test-table [id=d367aed0b5ae4f18a3173d89f7872ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.901120 29871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.901120 29663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.901135 30087 tablet_service.cc:1467] Processing CreateTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 (DEFAULT_TABLE table=test-table [id=d367aed0b5ae4f18a3173d89f7872ea7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:32.901465 30087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:32.904198 30222 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Bootstrap starting.
I20260809 16:38:32.904299 30224 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Bootstrap starting.
I20260809 16:38:32.904960 30224 tablet_bootstrap.cc:654] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.904959 30222 tablet_bootstrap.cc:654] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.905293 30222 log.cc:824] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.905293 30224 log.cc:824] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.905619 30226 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: Bootstrap starting.
I20260809 16:38:32.906042 30224 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: No bootstrap required, opened a new log
I20260809 16:38:32.906116 30224 ts_tablet_manager.cc:1397] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.906140 30222 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: No bootstrap required, opened a new log
I20260809 16:38:32.906206 30222 ts_tablet_manager.cc:1397] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.906231 30226 tablet_bootstrap.cc:654] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:32.906526 30226 log.cc:824] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:32.907362 30226 tablet_bootstrap.cc:492] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: No bootstrap required, opened a new log
I20260809 16:38:32.907426 30226 ts_tablet_manager.cc:1397] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:32.908325 30222 raft_consensus.cc:348] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.908459 30222 raft_consensus.cc:374] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.908491 30222 raft_consensus.cc:729] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aea725ec679344bba46c04eb0f96ed1f, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.908625 30222 consensus_queue.cc:260] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.908674 30224 raft_consensus.cc:348] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.908836 30224 raft_consensus.cc:374] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.908880 30224 raft_consensus.cc:729] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94c8f252d06b4fd5a080be493cdaacb3, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.908885 30222 ts_tablet_manager.cc:1428] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:32.908993 30224 consensus_queue.cc:260] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.909034 30198 heartbeater.cc:503] Master 127.31.15.254:46057 was elected leader, sending a full tablet report...
I20260809 16:38:32.909260 30224 ts_tablet_manager.cc:1428] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:32.909387 29978 heartbeater.cc:503] Master 127.31.15.254:46057 was elected leader, sending a full tablet report...
I20260809 16:38:32.909951 30226 raft_consensus.cc:348] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.910109 30226 raft_consensus.cc:374] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:32.910161 30226 raft_consensus.cc:729] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375a16bd1f5b416bb20eaf76b3a9a52c, State: Initialized, Role: FOLLOWER
I20260809 16:38:32.910274 30226 consensus_queue.cc:260] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.910511 30226 ts_tablet_manager.cc:1428] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:32.910552 29768 heartbeater.cc:503] Master 127.31.15.254:46057 was elected leader, sending a full tablet report...
I20260809 16:38:32.996495 30232 raft_consensus.cc:482] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:32.996631 30232 raft_consensus.cc:504] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:32.997021 30232 leader_election.cc:288] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aea725ec679344bba46c04eb0f96ed1f (127.31.15.195:39539), 375a16bd1f5b416bb20eaf76b3a9a52c (127.31.15.193:32823)
I20260809 16:38:32.999867 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.997259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:53853 (local address 127.31.15.195:39539)
0809 16:38:32.997316 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.997318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.997328 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:32.997361 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.997362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.997371 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.997430 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.997594 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.998109 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999095 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.999513 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999586 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.999594 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.999762 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.999800 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:38:32.999857 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.997177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:39539 (local address 127.31.15.194:53853)
0809 16:38:32.997238 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.997249 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.997292 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.997448 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.997449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.997451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.997529 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.997531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.998181 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.998182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.999015 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.999017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.999588 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.999657 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.999661 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.999751 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.999785 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 16:38:32.999924 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.997273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:50553 (local address 127.31.15.193:32823)
0809 16:38:32.997322 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:32.997323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:32.997429 (+ 106us) server_negotiation.cc:407] Connection header received
0809 16:38:32.997483 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:32.997484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:32.997495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:32.997562 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:32.997697 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.998155 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999117 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.999594 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999660 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.999668 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:32.999825 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 16:38:32.999865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:38:32.999965 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:32.997197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:32823 (local address 127.31.15.194:50553)
0809 16:38:32.997368 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:32.997378 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:32.997431 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:32.997560 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:32.997560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:32.997562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:32.997634 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.997636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.998234 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.998235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.999034 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:32.999036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:32.999671 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:32.999672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:32.999752 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:32.999756 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:32.999812 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 16:38:32.999850 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":18,"thread_start_us":93,"threads_started":1}
I20260809 16:38:33.000231 30124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df5c8a8fdef4d2b92e8fd97e9a264f0" candidate_uuid: "94c8f252d06b4fd5a080be493cdaacb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aea725ec679344bba46c04eb0f96ed1f" is_pre_election: true
I20260809 16:38:33.000387 30124 raft_consensus.cc:2454] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94c8f252d06b4fd5a080be493cdaacb3 in term 0.
I20260809 16:38:33.000479 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df5c8a8fdef4d2b92e8fd97e9a264f0" candidate_uuid: "94c8f252d06b4fd5a080be493cdaacb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" is_pre_election: true
I20260809 16:38:33.000625 29698 raft_consensus.cc:2454] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94c8f252d06b4fd5a080be493cdaacb3 in term 0.
I20260809 16:38:33.000619 29800 leader_election.cc:302] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94c8f252d06b4fd5a080be493cdaacb3, aea725ec679344bba46c04eb0f96ed1f; no voters:
I20260809 16:38:33.000725 30232 raft_consensus.cc:2781] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:33.000782 30232 raft_consensus.cc:482] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:33.000810 30232 raft_consensus.cc:3037] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:33.001483 30232 raft_consensus.cc:504] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:33.001607 30232 leader_election.cc:288] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [CANDIDATE]: Term 1 election: Requested vote from peers aea725ec679344bba46c04eb0f96ed1f (127.31.15.195:39539), 375a16bd1f5b416bb20eaf76b3a9a52c (127.31.15.193:32823)
I20260809 16:38:33.001747 30124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df5c8a8fdef4d2b92e8fd97e9a264f0" candidate_uuid: "94c8f252d06b4fd5a080be493cdaacb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aea725ec679344bba46c04eb0f96ed1f"
I20260809 16:38:33.001753 29698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df5c8a8fdef4d2b92e8fd97e9a264f0" candidate_uuid: "94c8f252d06b4fd5a080be493cdaacb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c"
I20260809 16:38:33.001818 30124 raft_consensus.cc:3037] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:33.001820 29698 raft_consensus.cc:3037] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:33.002480 30124 raft_consensus.cc:2454] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94c8f252d06b4fd5a080be493cdaacb3 in term 1.
I20260809 16:38:33.002590 29698 raft_consensus.cc:2454] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94c8f252d06b4fd5a080be493cdaacb3 in term 1.
I20260809 16:38:33.002628 29800 leader_election.cc:302] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94c8f252d06b4fd5a080be493cdaacb3, aea725ec679344bba46c04eb0f96ed1f; no voters:
I20260809 16:38:33.002768 30232 raft_consensus.cc:2781] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:33.002914 30232 raft_consensus.cc:686] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 1 LEADER]: Becoming Leader. State: Replica: 94c8f252d06b4fd5a080be493cdaacb3, State: Running, Role: LEADER
I20260809 16:38:33.003000 30232 consensus_queue.cc:237] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } }
I20260809 16:38:33.003877 29464 catalog_manager.cc:5495] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94c8f252d06b4fd5a080be493cdaacb3 (127.31.15.194). New cstate: current_term: 1 leader_uuid: "94c8f252d06b4fd5a080be493cdaacb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94c8f252d06b4fd5a080be493cdaacb3" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 38173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 } health_report { overall_health: UNKNOWN } } }
W20260809 16:38:33.004136 29981 tablet.cc:2369] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:33.013512 30221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:33.011520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.31.15.194:38173)
0809 16:38:33.011563 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:33.011565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:33.011572 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:38:33.011629 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:33.011630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:33.011645 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:33.011696 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:33.011787 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:33.012233 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:33.012893 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:33.013073 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:33.013120 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:33.013228 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:33.013309 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:33.013373 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:33.013408 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 16:38:33.013415 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:38:33.015138 30124 raft_consensus.cc:1261] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 1 FOLLOWER]: Refusing update from remote peer 94c8f252d06b4fd5a080be493cdaacb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:33.015156 29698 raft_consensus.cc:1261] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 375a16bd1f5b416bb20eaf76b3a9a52c [term 1 FOLLOWER]: Refusing update from remote peer 94c8f252d06b4fd5a080be493cdaacb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:33.015462 30232 consensus_queue.cc:1035] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "375a16bd1f5b416bb20eaf76b3a9a52c" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 32823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:33.015595 30238 consensus_queue.cc:1035] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aea725ec679344bba46c04eb0f96ed1f" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 39539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:33.017834 30242 mvcc.cc:205] Tried to move back new op lower bound from 7316658229307584512 to 7316658229260951552. Current Snapshot: MvccSnapshot[applied={T|T < 7316658229307584512}]
I20260809 16:38:33.017968 30245 mvcc.cc:205] Tried to move back new op lower bound from 7316658229307584512 to 7316658229260951552. Current Snapshot: MvccSnapshot[applied={T|T < 7316658229307584512}]
I20260809 16:38:33.018384 30243 mvcc.cc:205] Tried to move back new op lower bound from 7316658229307584512 to 7316658229260951552. Current Snapshot: MvccSnapshot[applied={T|T < 7316658229307584512}]
I20260809 16:38:33.020213 29462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:33.020291 29462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:33.020960 29462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77c0fe2b870e4c749cd08cfab6699f2c: Sending DeleteTablet for 3 replicas of tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0
I20260809 16:38:33.021271 30087 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:33 UTC) from {username='slave'} at 127.0.0.1:48070
I20260809 16:38:33.021363 29663 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:33 UTC) from {username='slave'} at 127.0.0.1:35048
I20260809 16:38:33.021353 29871 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:33 UTC) from {username='slave'} at 127.0.0.1:52984
I20260809 16:38:33.021541 30262 tablet_replica.cc:330] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: stopping tablet replica
I20260809 16:38:33.021697 30262 raft_consensus.cc:2229] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:33.021802 30262 raft_consensus.cc:2258] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:33.021914 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 29556
I20260809 16:38:33.030282 30264 tablet_replica.cc:330] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: stopping tablet replica
I20260809 16:38:33.030414 30264 raft_consensus.cc:2229] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:38:33.030548 30264 raft_consensus.cc:2258] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:33.031584 30264 ts_tablet_manager.cc:1905] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:33.033288 30264 ts_tablet_manager.cc:1918] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:33.033371 30264 log.cc:1196] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/9df5c8a8fdef4d2b92e8fd97e9a264f0
I20260809 16:38:33.033632 30264 ts_tablet_manager.cc:1939] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P 94c8f252d06b4fd5a080be493cdaacb3: Deleting consensus metadata
I20260809 16:38:33.033990 30262 ts_tablet_manager.cc:1905] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:33.034129 29442 catalog_manager.cc:4841] TS 94c8f252d06b4fd5a080be493cdaacb3 (127.31.15.194:38173): tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 (table test-table [id=d367aed0b5ae4f18a3173d89f7872ea7]) successfully deleted
I20260809 16:38:33.035735 30262 ts_tablet_manager.cc:1918] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:33.035815 30262 log.cc:1196] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/9df5c8a8fdef4d2b92e8fd97e9a264f0
I20260809 16:38:33.036046 30262 ts_tablet_manager.cc:1939] T 9df5c8a8fdef4d2b92e8fd97e9a264f0 P aea725ec679344bba46c04eb0f96ed1f: Deleting consensus metadata
W20260809 16:38:33.036242 29439 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:32823 (error 108)
I20260809 16:38:33.036473 29437 catalog_manager.cc:4841] TS aea725ec679344bba46c04eb0f96ed1f (127.31.15.195:39539): tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0 (table test-table [id=d367aed0b5ae4f18a3173d89f7872ea7]) successfully deleted
I20260809 16:38:33.036759 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:33.036643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:32823 (local address 127.0.0.1:35060)
0809 16:38:33.036685 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:33.036729 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 16:38:33.036864 29439 catalog_manager.cc:4584] TS 375a16bd1f5b416bb20eaf76b3a9a52c (127.31.15.193:32823): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9df5c8a8fdef4d2b92e8fd97e9a264f0: Network error: Client connection negotiation failed: client connection to 127.31.15.193:32823: connect: Connection refused (error 111)
I20260809 16:38:33.037106 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 29773
I20260809 16:38:33.049180 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 29984
I20260809 16:38:33.057209 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 29418
2026-08-09T16:38:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (763 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:33 dist-test-slave-21b9 krb5kdc[30268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:33 dist-test-slave-21b9 krb5kdc[30268](info): set up 2 sockets
Aug 09 16:38:33 dist-test-slave-21b9 krb5kdc[30268](info): commencing operation
krb5kdc: starting...
W20260809 16:38:34.212931 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:34.324033 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40397
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:34.403748 30290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:34.403940 30290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:34.403962 30290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:34.405077 30290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:34.405119 30290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:34.405133 30290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:34.405143 30290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:34.405165 30290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:34.406635 30290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:34.406864 30290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:34.407066 30290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:34.411608 30296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.411671 30290 server_base.cc:1034] running on GCE node
W20260809 16:38:34.411643 30297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:34.411629 30299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.412002 30290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:34.412258 30290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:34.413425 30290 hybrid_clock.cc:648] HybridClock initialized: now 1786293514413410 us; error 31 us; skew 500 ppm
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:34.415771 30290 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:34.416195 30290 webserver.cc:460] Webserver started at http://127.31.15.254:43923/ using document root <none> and password file <none>
I20260809 16:38:34.416396 30290 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:34.416441 30290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:34.416563 30290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:34.417544 30290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "22aee6ad19ef4a79ac96f5bb500e344e"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d80e690682145af0fa3a95122aaa01ba"
server_key_iv: "0dae5f96c1079cbbf254419da1780fe4"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.417860 30290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "22aee6ad19ef4a79ac96f5bb500e344e"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d80e690682145af0fa3a95122aaa01ba"
server_key_iv: "0dae5f96c1079cbbf254419da1780fe4"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.419080 30290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:34.419754 30309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.419888 30290 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.419977 30290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "22aee6ad19ef4a79ac96f5bb500e344e"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d80e690682145af0fa3a95122aaa01ba"
server_key_iv: "0dae5f96c1079cbbf254419da1780fe4"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.420043 30290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:34.434934 30290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:34.436724 30290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:34.436856 30290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:34.440732 30290 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40397
I20260809 16:38:34.440784 30397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40397 every 8 connection(s)
I20260809 16:38:34.441097 30290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:34.441917 30398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:34.444129 30398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: Bootstrap starting.
I20260809 16:38:34.444656 30398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:34.444886 30398 log.cc:824] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:34.445537 30398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: No bootstrap required, opened a new log
I20260809 16:38:34.447113 30398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } }
I20260809 16:38:34.447253 30398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:34.447328 30398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22aee6ad19ef4a79ac96f5bb500e344e, State: Initialized, Role: FOLLOWER
I20260809 16:38:34.447474 30398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } }
I20260809 16:38:34.447538 30398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:34.447562 30398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:34.447607 30398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:34.448115 30398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } }
I20260809 16:38:34.448204 30398 leader_election.cc:302] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22aee6ad19ef4a79ac96f5bb500e344e; no voters:
I20260809 16:38:34.448362 30398 leader_election.cc:288] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:34.448403 30403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:34.448457 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 30290
I20260809 16:38:34.448534 30403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [term 1 LEADER]: Becoming Leader. State: Replica: 22aee6ad19ef4a79ac96f5bb500e344e, State: Running, Role: LEADER
I20260809 16:38:34.448576 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:34.448617 30403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } }
I20260809 16:38:34.448624 30398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:34.448783 31807 external_mini_cluster.cc:1428] Setting key f224432ca83e70dad010bf3800802b90
I20260809 16:38:34.449914 30404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } } }
I20260809 16:38:34.449911 30406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22aee6ad19ef4a79ac96f5bb500e344e. Latest consensus state: current_term: 1 leader_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22aee6ad19ef4a79ac96f5bb500e344e" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40397 } } }
I20260809 16:38:34.450042 30404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:34.450040 30406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e [sys.catalog]: This master's current role is: LEADER
W20260809 16:38:34.450480 30432 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:38:34.450575 30432 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:38:34.450630 30435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:34.450779 30435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:34.452240 30435 catalog_manager.cc:1347] Generated new cluster ID: 2ecef9e022634ebfa6a1b19769821571
I20260809 16:38:34.452286 30435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:34.460834 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.449250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.31.15.254:40397)
0809 16:38:34.449446 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.449457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.449491 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:38:34.449615 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.449622 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.450582 (+ 960us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.450764 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:34.451432 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.452803 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.453861 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.454469 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.454591 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.457346 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:34.457350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:34.457351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:34.457363 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:34.459215 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.459627 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.459630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.459630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.459740 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.459932 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.459933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.459933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.460226 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:34.460363 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.460468 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.460601 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260809 16:38:34.461959 30435 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:34.462426 30435 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:34.467690 30435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: Generated new TSK 0
I20260809 16:38:34.467846 30435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.31.15.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
Entry for principal kudu/127.31.15.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab.
I20260809 16:38:34.496488 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:34.578019 30442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:34.578220 30442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:34.578250 30442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:34.579416 30442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:34.579455 30442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:34.579491 30442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:34.580929 30442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.193.keytab
--principal=kudu/127.31.15.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:34.581158 30442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:34.581370 30442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:34.586148 30452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:34.586174 30448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:34.586210 30449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.586395 30442 server_base.cc:1034] running on GCE node
I20260809 16:38:34.586571 30442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:34.586772 30442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:34.587901 30442 hybrid_clock.cc:648] HybridClock initialized: now 1786293514587887 us; error 32 us; skew 500 ppm
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:34.590301 30442 init.cc:377] Logged in from keytab as kudu/127.31.15.193@KRBTEST.COM (short username kudu)
I20260809 16:38:34.590693 30442 webserver.cc:460] Webserver started at http://127.31.15.193:46699/ using document root <none> and password file <none>
I20260809 16:38:34.590883 30442 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:34.590926 30442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:34.591040 30442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:34.592059 30442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "7b89fb8286de44baad395538017c9b3d"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d2f11c78b5429f8691cedc19c0cbe81e"
server_key_iv: "35b8cb75fc9f2ee0775e225352a13db1"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.592386 30442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "7b89fb8286de44baad395538017c9b3d"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d2f11c78b5429f8691cedc19c0cbe81e"
server_key_iv: "35b8cb75fc9f2ee0775e225352a13db1"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.593613 30442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:38:34.594275 30463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.594412 30442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:34.594496 30442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "7b89fb8286de44baad395538017c9b3d"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "d2f11c78b5429f8691cedc19c0cbe81e"
server_key_iv: "35b8cb75fc9f2ee0775e225352a13db1"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.594555 30442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:34.606240 30442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:34.608006 30442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:34.608147 30442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:34.608353 30442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:34.608637 30442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:34.608671 30442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.608700 30442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:34.608716 30442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.614198 30442 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:45619
I20260809 16:38:34.614300 30648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:45619 every 8 connection(s)
I20260809 16:38:34.614555 30442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.193@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:34.621459 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 30442
I20260809 16:38:34.621587 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:34.621793 31807 external_mini_cluster.cc:1428] Setting key f8db36529f68b5acbbe4f633eae1c234
I20260809 16:38:34.623486 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.615253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:46459 (local address 127.31.15.254:40397)
0809 16:38:34.615389 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.615391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.615521 (+ 130us) server_negotiation.cc:407] Connection header received
0809 16:38:34.616031 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.616032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.616048 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.616101 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:34.617042 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.617556 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.618514 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.618723 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.618778 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.620813 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:34.620814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:34.620814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:34.620822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:34.622138 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.622527 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.622528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.622528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.622590 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.622860 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.622861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.622861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.622971 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:34.623021 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.623347 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.623415 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 16:38:34.623719 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.615197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40397 (local address 127.31.15.193:46459)
0809 16:38:34.615370 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.615420 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.615932 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.616170 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.616173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.616757 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:34.616944 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.616950 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.617638 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.617639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.618423 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.618426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.618793 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.618794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.618909 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.619351 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:34.619358 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:34.620717 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:34.622202 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.622208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.622210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.622467 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.622652 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.622652 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.622653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.622792 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.623026 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:34.623032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:34.623197 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.623329 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.623502 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260809 16:38:34.624187 30650 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40397
I20260809 16:38:34.624291 30650 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:34.624485 30650 heartbeater.cc:511] Master 127.31.15.254:40397 requested a full tablet report, sending...
I20260809 16:38:34.624995 30333 ts_manager.cc:194] Registered new tserver with Master: 7b89fb8286de44baad395538017c9b3d (127.31.15.193:45619)
I20260809 16:38:34.625599 30333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.193@KRBTEST.COM'} at 127.31.15.193:46459
WARNING: no policy specified for kudu/127.31.15.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
Entry for principal kudu/127.31.15.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab.
I20260809 16:38:34.645687 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:34.725893 30656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:34.726081 30656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:34.726104 30656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:34.727214 30656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:34.727257 30656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:34.727303 30656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:34.728744 30656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.194.keytab
--principal=kudu/127.31.15.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:34.728965 30656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:34.729193 30656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:34.733923 30662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.734035 30656 server_base.cc:1034] running on GCE node
W20260809 16:38:34.733911 30667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:34.733935 30664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.734329 30656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:34.734557 30656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:34.735705 30656 hybrid_clock.cc:648] HybridClock initialized: now 1786293514735693 us; error 29 us; skew 500 ppm
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:34.738061 30656 init.cc:377] Logged in from keytab as kudu/127.31.15.194@KRBTEST.COM (short username kudu)
I20260809 16:38:34.738446 30656 webserver.cc:460] Webserver started at http://127.31.15.194:36305/ using document root <none> and password file <none>
I20260809 16:38:34.738663 30656 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:34.738710 30656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:34.738832 30656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:34.739840 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "5d37cfa425b2401997adf0b63ff1d969"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "ac57fbfb11adbb618195fa0508f3dbd2"
server_key_iv: "4fe4d1eb37d34d2ddfee66c7afdf0354"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.740159 30656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "5d37cfa425b2401997adf0b63ff1d969"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "ac57fbfb11adbb618195fa0508f3dbd2"
server_key_iv: "4fe4d1eb37d34d2ddfee66c7afdf0354"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.741381 30656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:34.742079 30675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.742216 30656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:34.742300 30656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "5d37cfa425b2401997adf0b63ff1d969"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "ac57fbfb11adbb618195fa0508f3dbd2"
server_key_iv: "4fe4d1eb37d34d2ddfee66c7afdf0354"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.742357 30656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:34.754887 30656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:34.756659 30656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:34.756790 30656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:34.756991 30656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:34.757272 30656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:34.757304 30656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.757331 30656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:34.757349 30656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.762538 30656 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:40589
I20260809 16:38:34.762605 30872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:40589 every 8 connection(s)
I20260809 16:38:34.762866 30656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.194@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:34.770239 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 30656
I20260809 16:38:34.770367 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:34.770576 31807 external_mini_cluster.cc:1428] Setting key 867dd1d13b87914babbfd02f22d9f1f8
I20260809 16:38:34.771776 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.763583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:57725 (local address 127.31.15.254:40397)
0809 16:38:34.763655 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.763657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.763809 (+ 152us) server_negotiation.cc:407] Connection header received
0809 16:38:34.764305 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.764306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.764318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.764366 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:34.765300 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.765798 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.766773 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.766987 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.767043 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.769047 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:34.769048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:34.769049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:34.769056 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:34.770382 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.770835 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.770837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.770837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.770904 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.771166 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.771167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.771167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.771314 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:34.771376 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.771614 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.771694 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 16:38:34.771862 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.763517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40397 (local address 127.31.15.194:57725)
0809 16:38:34.763688 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.763732 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.764204 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.764478 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.764482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.765020 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:34.765204 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.765208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.765879 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.765880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.766689 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.766691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.767055 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.767056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.767176 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.767646 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:34.767655 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:34.768952 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:34.770477 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.770485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.770487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.770753 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.770969 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.770970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.770970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.771104 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.771374 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:34.771380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:34.771500 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.771578 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.771695 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260809 16:38:34.772286 30873 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40397
I20260809 16:38:34.772374 30873 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:34.772547 30873 heartbeater.cc:511] Master 127.31.15.254:40397 requested a full tablet report, sending...
I20260809 16:38:34.772897 30333 ts_manager.cc:194] Registered new tserver with Master: 5d37cfa425b2401997adf0b63ff1d969 (127.31.15.194:40589)
I20260809 16:38:34.773301 30333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.194@KRBTEST.COM'} at 127.31.15.194:57725
WARNING: no policy specified for kudu/127.31.15.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
Entry for principal kudu/127.31.15.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab.
I20260809 16:38:34.794463 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:34.878244 30884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:34.878444 30884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:34.878465 30884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:34.879596 30884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:34.879629 30884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:34.879654 30884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:34.881052 30884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.195.keytab
--principal=kudu/127.31.15.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:34.881244 30884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:34.881453 30884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:34.886204 30892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:34.886250 30897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.886337 30884 server_base.cc:1034] running on GCE node
W20260809 16:38:34.886214 30893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:34.886659 30884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:34.886888 30884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:34.888026 30884 hybrid_clock.cc:648] HybridClock initialized: now 1786293514888012 us; error 31 us; skew 500 ppm
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:34.890309 30884 init.cc:377] Logged in from keytab as kudu/127.31.15.195@KRBTEST.COM (short username kudu)
I20260809 16:38:34.890729 30884 webserver.cc:460] Webserver started at http://127.31.15.195:43015/ using document root <none> and password file <none>
I20260809 16:38:34.890938 30884 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:34.890985 30884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:34.891108 30884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:34.892117 30884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "4b248d9dd041425b8e4e3d0658f04562"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "920cc78e2daf546c3bd6baa38833fe38"
server_key_iv: "abb58ea61d5de595ae20040171c55450"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.892438 30884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "4b248d9dd041425b8e4e3d0658f04562"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "920cc78e2daf546c3bd6baa38833fe38"
server_key_iv: "abb58ea61d5de595ae20040171c55450"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.893620 30884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:34.894240 30908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.894385 30884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:34.894465 30884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "4b248d9dd041425b8e4e3d0658f04562"
format_stamp: "Formatted at 2026-08-09 16:38:34 on dist-test-slave-21b9"
server_key: "920cc78e2daf546c3bd6baa38833fe38"
server_key_iv: "abb58ea61d5de595ae20040171c55450"
server_key_version: "encryptionkey@0"
I20260809 16:38:34.894536 30884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:34.934191 30884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:34.936014 30884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:34.936149 30884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:34.936365 30884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:34.936671 30884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:34.936707 30884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.936736 30884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:34.936759 30884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:34.942121 30884 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:45785
I20260809 16:38:34.942188 31109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:45785 every 8 connection(s)
I20260809 16:38:34.942469 30884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
Aug 09 16:38:34 dist-test-slave-21b9 krb5kdc[30268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293514, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.195@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:34.951151 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.943161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:38267 (local address 127.31.15.254:40397)
0809 16:38:34.943222 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.943224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.943401 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:38:34.943875 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.943876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.943886 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.943926 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:34.944831 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.945325 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.946358 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.946550 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.946606 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.948536 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:34.948538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:34.948538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:34.948545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:34.949842 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.950234 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.950236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.950236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.950312 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.950537 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.950538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.950539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.950654 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:34.950711 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.950982 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.951061 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:38:34.951349 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.943108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40397 (local address 127.31.15.195:38267)
0809 16:38:34.943258 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.943321 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.943785 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.943981 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.943985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.944544 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:34.944737 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.944742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.945424 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.945426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.946288 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.946291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.946617 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.946618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.946712 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.947108 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:38:34.947116 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:34.948445 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:34.949934 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.949941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.949943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.950165 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.950375 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:38:34.950376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:38:34.950377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:38:34.950480 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:38:34.950711 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:34.950718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:34.950850 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.950945 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.951112 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260809 16:38:34.951798 31111 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40397
I20260809 16:38:34.951884 31111 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:34.952056 31111 heartbeater.cc:511] Master 127.31.15.254:40397 requested a full tablet report, sending...
I20260809 16:38:34.952301 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 30884
I20260809 16:38:34.952430 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:34.952452 30333 ts_manager.cc:194] Registered new tserver with Master: 4b248d9dd041425b8e4e3d0658f04562 (127.31.15.195:45785)
I20260809 16:38:34.952631 31807 external_mini_cluster.cc:1428] Setting key b826eda407857e4611fc9089a219d412
I20260809 16:38:34.952844 30333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.31.15.195@KRBTEST.COM'} at 127.31.15.195:38267
I20260809 16:38:34.953490 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:34.959894 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.954748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57018 (local address 127.31.15.254:40397)
0809 16:38:34.954800 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.954802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.954890 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:38:34.954944 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.954945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.954953 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.954999 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:34.955601 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.956041 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.956729 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.956910 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.956962 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.957939 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:34.957940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:34.957940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:34.957947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:34.958886 (+ 939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.959182 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.959183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.959183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.959232 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:34.959402 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:34.959403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:34.959403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:34.959532 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:34.959630 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.959743 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.959813 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:38:34.961364 30328 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:34.962838 30328 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:34.968765 31131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.964965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:45785 (local address 127.0.0.1:40758)
0809 16:38:34.965278 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.965287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.965338 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.965673 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.965674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.965676 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:34.965713 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.965714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966889 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.966890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.967758 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.967761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.968482 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.968483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.968560 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.968574 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.968639 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.968692 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":38,"threads_started":1}
I20260809 16:38:34.968773 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.965049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.31.15.195:45785)
0809 16:38:34.965209 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.965216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.965393 (+ 177us) server_negotiation.cc:407] Connection header received
0809 16:38:34.965433 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.965435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.965450 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.965610 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:34.965781 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966795 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.967828 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.968406 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.968470 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.968527 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.968659 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.968696 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260809 16:38:34.968932 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.964895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56824 (local address 127.31.15.193:45619)
0809 16:38:34.965238 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.965243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.965346 (+ 103us) server_negotiation.cc:407] Connection header received
0809 16:38:34.965398 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.965401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.965415 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.965484 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:34.965720 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966417 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.967770 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.968450 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.968517 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.968572 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.968786 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.968841 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":77,"threads_started":1}
I20260809 16:38:34.969143 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.964764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45619 (local address 127.0.0.1:56824)
0809 16:38:34.965264 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.965274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.965338 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.965578 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.965579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.965582 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:34.965634 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.965636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966508 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.966508 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.967692 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.967695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.968619 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.968620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.968684 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.968688 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.969018 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.969064 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":69,"threads_started":1}
I20260809 16:38:34.969489 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.964761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:40589 (local address 127.0.0.1:44104)
0809 16:38:34.964880 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:34.964918 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:34.964974 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:34.965446 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:34.965449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:34.965451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:34.965515 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.965517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966844 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.966846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.967951 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:34.967953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.969327 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.969328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:34.969380 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.969383 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:34.969404 (+ 21us) client_negotiation.cc:240] Negotiation successful
0809 16:38:34.969433 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":10,"thread_start_us":53,"threads_started":1}
I20260809 16:38:34.969527 30989 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2fb28837624fc7b264986613b5baf7 (DEFAULT_TABLE table=test-table [id=bc1869dc2dba485694da78339fb435cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:34.969872 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:34.965091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.31.15.194:40589)
0809 16:38:34.965263 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:34.965267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:34.965277 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:34.965311 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:34.965312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:34.965326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:34.965440 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:34.965605 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.966733 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.968041 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:34.968732 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:34.969658 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:34.969723 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:34.969748 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 16:38:34.969791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 16:38:34.969884 30989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2fb28837624fc7b264986613b5baf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:34.969929 30543 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2fb28837624fc7b264986613b5baf7 (DEFAULT_TABLE table=test-table [id=bc1869dc2dba485694da78339fb435cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:34.970268 30543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2fb28837624fc7b264986613b5baf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:34.970471 30764 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2fb28837624fc7b264986613b5baf7 (DEFAULT_TABLE table=test-table [id=bc1869dc2dba485694da78339fb435cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:34.970798 30764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2fb28837624fc7b264986613b5baf7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:34.972683 31136 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: Bootstrap starting.
I20260809 16:38:34.973237 31137 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Bootstrap starting.
I20260809 16:38:34.973349 31136 tablet_bootstrap.cc:654] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:34.973814 31138 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Bootstrap starting.
I20260809 16:38:34.973862 31137 tablet_bootstrap.cc:654] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:34.974095 31136 log.cc:824] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:34.974376 31137 log.cc:824] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:34.974398 31138 tablet_bootstrap.cc:654] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:34.974951 31136 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: No bootstrap required, opened a new log
I20260809 16:38:34.974942 31138 log.cc:824] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:34.975006 31136 ts_tablet_manager.cc:1397] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:38:34.975190 31137 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: No bootstrap required, opened a new log
I20260809 16:38:34.975257 31137 ts_tablet_manager.cc:1397] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 16:38:34.975677 31138 tablet_bootstrap.cc:492] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: No bootstrap required, opened a new log
I20260809 16:38:34.975747 31138 ts_tablet_manager.cc:1397] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:34.976611 31136 raft_consensus.cc:348] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.976748 31136 raft_consensus.cc:374] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:34.976783 31136 raft_consensus.cc:729] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b89fb8286de44baad395538017c9b3d, State: Initialized, Role: FOLLOWER
I20260809 16:38:34.976893 31136 consensus_queue.cc:260] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.977154 31136 ts_tablet_manager.cc:1428] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:38:34.977134 31137 raft_consensus.cc:348] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.977305 30650 heartbeater.cc:503] Master 127.31.15.254:40397 was elected leader, sending a full tablet report...
I20260809 16:38:34.977304 31137 raft_consensus.cc:374] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:34.977344 31137 raft_consensus.cc:729] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b248d9dd041425b8e4e3d0658f04562, State: Initialized, Role: FOLLOWER
I20260809 16:38:34.977455 31137 consensus_queue.cc:260] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.977598 31138 raft_consensus.cc:348] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.977736 31137 ts_tablet_manager.cc:1428] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 16:38:34.977766 31138 raft_consensus.cc:374] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:34.977808 31138 raft_consensus.cc:729] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d37cfa425b2401997adf0b63ff1d969, State: Initialized, Role: FOLLOWER
I20260809 16:38:34.977839 31111 heartbeater.cc:503] Master 127.31.15.254:40397 was elected leader, sending a full tablet report...
I20260809 16:38:34.977919 31138 consensus_queue.cc:260] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:34.978207 31138 ts_tablet_manager.cc:1428] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 16:38:34.978303 30873 heartbeater.cc:503] Master 127.31.15.254:40397 was elected leader, sending a full tablet report...
I20260809 16:38:35.004666 31146 raft_consensus.cc:482] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:35.004788 31146 raft_consensus.cc:504] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:35.005162 31146 leader_election.cc:288] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d37cfa425b2401997adf0b63ff1d969 (127.31.15.194:40589), 4b248d9dd041425b8e4e3d0658f04562 (127.31.15.195:45785)
I20260809 16:38:35.007853 30652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.005334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:40589 (local address 127.31.15.193:38563)
0809 16:38:35.005388 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.005401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.005451 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.005617 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.005618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.005619 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.005682 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.005684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.006288 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.006288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.007049 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.007052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.007614 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.007692 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.007695 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.007741 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.007779 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":4}
I20260809 16:38:35.007853 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.005419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:38563 (local address 127.31.15.194:40589)
0809 16:38:35.005478 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.005480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.005490 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:35.005520 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.005521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.005530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.005581 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.005745 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.006215 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007136 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.007547 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007611 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.007620 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.007752 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.007787 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:38:35.008006 31133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.005545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:37323 (local address 127.31.15.195:45785)
0809 16:38:35.005612 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.005614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.005692 (+ 78us) server_negotiation.cc:407] Connection header received
0809 16:38:35.005745 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.005746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.005755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.005801 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.005943 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.006392 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007328 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.007701 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007763 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.007770 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.007911 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.007945 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:38:35.008015 31150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.005451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:45785 (local address 127.31.15.193:37323)
0809 16:38:35.005629 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.005639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.005695 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.005816 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.005816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.005818 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.005888 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.005890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.006468 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.006469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.007223 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.007225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.007779 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.007780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.007850 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.007854 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.007899 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.007937 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
I20260809 16:38:35.008236 30796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2fb28837624fc7b264986613b5baf7" candidate_uuid: "7b89fb8286de44baad395538017c9b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d37cfa425b2401997adf0b63ff1d969" is_pre_election: true
I20260809 16:38:35.008312 31019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2fb28837624fc7b264986613b5baf7" candidate_uuid: "7b89fb8286de44baad395538017c9b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b248d9dd041425b8e4e3d0658f04562" is_pre_election: true
I20260809 16:38:35.008406 30796 raft_consensus.cc:2454] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b89fb8286de44baad395538017c9b3d in term 0.
I20260809 16:38:35.008466 31019 raft_consensus.cc:2454] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b89fb8286de44baad395538017c9b3d in term 0.
I20260809 16:38:35.008647 30466 leader_election.cc:302] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d37cfa425b2401997adf0b63ff1d969, 7b89fb8286de44baad395538017c9b3d; no voters:
I20260809 16:38:35.008782 31146 raft_consensus.cc:2781] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:35.008838 31146 raft_consensus.cc:482] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:35.008862 31146 raft_consensus.cc:3037] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.009563 31146 raft_consensus.cc:504] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:35.009677 31146 leader_election.cc:288] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [CANDIDATE]: Term 1 election: Requested vote from peers 5d37cfa425b2401997adf0b63ff1d969 (127.31.15.194:40589), 4b248d9dd041425b8e4e3d0658f04562 (127.31.15.195:45785)
I20260809 16:38:35.009841 31019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2fb28837624fc7b264986613b5baf7" candidate_uuid: "7b89fb8286de44baad395538017c9b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b248d9dd041425b8e4e3d0658f04562"
I20260809 16:38:35.009856 30796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2fb28837624fc7b264986613b5baf7" candidate_uuid: "7b89fb8286de44baad395538017c9b3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d37cfa425b2401997adf0b63ff1d969"
I20260809 16:38:35.009912 31019 raft_consensus.cc:3037] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.009915 30796 raft_consensus.cc:3037] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.010597 31019 raft_consensus.cc:2454] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b89fb8286de44baad395538017c9b3d in term 1.
I20260809 16:38:35.010617 30796 raft_consensus.cc:2454] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b89fb8286de44baad395538017c9b3d in term 1.
I20260809 16:38:35.010751 30466 leader_election.cc:302] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b248d9dd041425b8e4e3d0658f04562, 7b89fb8286de44baad395538017c9b3d; no voters:
I20260809 16:38:35.010836 31146 raft_consensus.cc:2781] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:35.011003 31146 raft_consensus.cc:686] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [term 1 LEADER]: Becoming Leader. State: Replica: 7b89fb8286de44baad395538017c9b3d, State: Running, Role: LEADER
I20260809 16:38:35.011093 31146 consensus_queue.cc:237] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } }
I20260809 16:38:35.011972 30328 catalog_manager.cc:5495] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b89fb8286de44baad395538017c9b3d (127.31.15.193). New cstate: current_term: 1 leader_uuid: "7b89fb8286de44baad395538017c9b3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b89fb8286de44baad395538017c9b3d" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 45619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 } health_report { overall_health: UNKNOWN } } }
W20260809 16:38:35.013392 30874 tablet.cc:2369] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:35.028347 31128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.026051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.31.15.193:45619)
0809 16:38:35.026099 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.026101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.026110 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:38:35.026170 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.026171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.026189 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.026246 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:35.026375 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.026834 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.027634 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.027831 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.027885 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.028040 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:35.028143 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:35.028212 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.028249 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.028271 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:38:35.030112 30796 raft_consensus.cc:1261] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 1 FOLLOWER]: Refusing update from remote peer 7b89fb8286de44baad395538017c9b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:35.030122 31019 raft_consensus.cc:1261] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 1 FOLLOWER]: Refusing update from remote peer 7b89fb8286de44baad395538017c9b3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:35.030455 31152 consensus_queue.cc:1035] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b248d9dd041425b8e4e3d0658f04562" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 45785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:35.030588 31146 consensus_queue.cc:1035] T 3f2fb28837624fc7b264986613b5baf7 P 7b89fb8286de44baad395538017c9b3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d37cfa425b2401997adf0b63ff1d969" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:35.034881 30333 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:35.034967 30333 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:35.035733 30333 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22aee6ad19ef4a79ac96f5bb500e344e: Sending DeleteTablet for 3 replicas of tablet 3f2fb28837624fc7b264986613b5baf7
I20260809 16:38:35.036051 30543 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2fb28837624fc7b264986613b5baf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:56824
I20260809 16:38:35.036324 30989 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2fb28837624fc7b264986613b5baf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:40758
I20260809 16:38:35.036316 30764 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2fb28837624fc7b264986613b5baf7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='kudu', principal='kudu/127.31.15.254@KRBTEST.COM'} at 127.0.0.1:44104
I20260809 16:38:35.036552 31180 tablet_replica.cc:330] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: stopping tablet replica
I20260809 16:38:35.036608 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 30442
I20260809 16:38:35.039407 31180 raft_consensus.cc:2229] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:35.039608 31180 raft_consensus.cc:2258] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:35.041699 31179 tablet_replica.cc:330] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: stopping tablet replica
I20260809 16:38:35.041875 31179 raft_consensus.cc:2229] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:35.042035 31179 raft_consensus.cc:2258] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:35.042555 31180 ts_tablet_manager.cc:1905] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:35.044183 31180 ts_tablet_manager.cc:1918] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:35.044257 31180 log.cc:1196] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/3f2fb28837624fc7b264986613b5baf7
I20260809 16:38:35.044526 31180 ts_tablet_manager.cc:1939] T 3f2fb28837624fc7b264986613b5baf7 P 5d37cfa425b2401997adf0b63ff1d969: Deleting consensus metadata
I20260809 16:38:35.044950 30312 catalog_manager.cc:4841] TS 5d37cfa425b2401997adf0b63ff1d969 (127.31.15.194:40589): tablet 3f2fb28837624fc7b264986613b5baf7 (table test-table [id=bc1869dc2dba485694da78339fb435cf]) successfully deleted
I20260809 16:38:35.046545 31179 ts_tablet_manager.cc:1905] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:35.047752 31179 ts_tablet_manager.cc:1918] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:35.047821 31179 log.cc:1196] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/3f2fb28837624fc7b264986613b5baf7
I20260809 16:38:35.048064 31179 ts_tablet_manager.cc:1939] T 3f2fb28837624fc7b264986613b5baf7 P 4b248d9dd041425b8e4e3d0658f04562: Deleting consensus metadata
I20260809 16:38:35.048424 30312 catalog_manager.cc:4841] TS 4b248d9dd041425b8e4e3d0658f04562 (127.31.15.195:45785): tablet 3f2fb28837624fc7b264986613b5baf7 (table test-table [id=bc1869dc2dba485694da78339fb435cf]) successfully deleted
W20260809 16:38:35.051546 30316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:45619 (error 108)
I20260809 16:38:35.051872 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 30656
I20260809 16:38:35.067919 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.067684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:45619 (local address 127.0.0.1:56840)
0809 16:38:35.067774 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.067853 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 16:38:35.068666 30316 catalog_manager.cc:4584] TS 7b89fb8286de44baad395538017c9b3d (127.31.15.193:45619): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f2fb28837624fc7b264986613b5baf7: Network error: Client connection negotiation failed: client connection to 127.31.15.193:45619: connect: Connection refused (error 111)
I20260809 16:38:35.072286 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 30884
I20260809 16:38:35.095538 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 30290
2026-08-09T16:38:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2038 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T16:38:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:35Z Disabled control of system clock
I20260809 16:38:35.128327 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40353
--webserver_interface=127.31.15.254
--webserver_port=0
--builtin_ntp_servers=127.31.15.212:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:35.208575 31192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:35.208794 31192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:35.208813 31192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:35.209895 31192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:35.209932 31192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:35.209944 31192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:35.209955 31192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:35.209967 31192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:35.211225 31192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:35.211465 31192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:35.211721 31192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:35.216666 31204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.216586 31202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.216586 31201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:35.217201 31192 server_base.cc:1034] running on GCE node
I20260809 16:38:35.217478 31192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:35.217819 31192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:35.218972 31192 hybrid_clock.cc:648] HybridClock initialized: now 1786293515218953 us; error 35 us; skew 500 ppm
I20260809 16:38:35.220191 31192 webserver.cc:460] Webserver started at http://127.31.15.254:36103/ using document root <none> and password file <none>
I20260809 16:38:35.220419 31192 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:35.220469 31192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:35.220604 31192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:35.221607 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "b1dc78a2f9b1426b8805a570ccca8549"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "cf59babe48f52639d7469e90eba38346"
server_key_iv: "e42133a1cee5583962c20660cfb97606"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.221932 31192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "b1dc78a2f9b1426b8805a570ccca8549"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "cf59babe48f52639d7469e90eba38346"
server_key_iv: "e42133a1cee5583962c20660cfb97606"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.223136 31192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:35.223934 31216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.224110 31192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:35.224197 31192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "b1dc78a2f9b1426b8805a570ccca8549"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "cf59babe48f52639d7469e90eba38346"
server_key_iv: "e42133a1cee5583962c20660cfb97606"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.224261 31192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:35.240967 31192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:35.241241 31192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:35.241366 31192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:35.245136 31192 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40353
I20260809 16:38:35.245169 31303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40353 every 8 connection(s)
I20260809 16:38:35.245473 31192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:35.246390 31304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:35.248600 31304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: Bootstrap starting.
I20260809 16:38:35.249141 31304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:35.249378 31304 log.cc:824] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:35.249974 31304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: No bootstrap required, opened a new log
I20260809 16:38:35.251536 31304 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } }
I20260809 16:38:35.251669 31304 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:35.251698 31304 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1dc78a2f9b1426b8805a570ccca8549, State: Initialized, Role: FOLLOWER
I20260809 16:38:35.251802 31304 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } }
I20260809 16:38:35.251850 31304 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:35.251870 31304 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:35.251894 31304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.252419 31304 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } }
I20260809 16:38:35.252435 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31192
I20260809 16:38:35.252530 31304 leader_election.cc:302] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1dc78a2f9b1426b8805a570ccca8549; no voters:
I20260809 16:38:35.252557 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:35.252696 31304 leader_election.cc:288] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:35.252743 31307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:35.252774 31807 external_mini_cluster.cc:1428] Setting key e573909462df0c13fd6cb4bac189a96c
I20260809 16:38:35.252899 31307 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [term 1 LEADER]: Becoming Leader. State: Replica: b1dc78a2f9b1426b8805a570ccca8549, State: Running, Role: LEADER
I20260809 16:38:35.252939 31304 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:35.253007 31307 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } }
I20260809 16:38:35.253305 31307 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1dc78a2f9b1426b8805a570ccca8549. Latest consensus state: current_term: 1 leader_uuid: "b1dc78a2f9b1426b8805a570ccca8549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } } }
I20260809 16:38:35.253381 31307 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:35.253502 31310 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1dc78a2f9b1426b8805a570ccca8549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1dc78a2f9b1426b8805a570ccca8549" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40353 } } }
I20260809 16:38:35.253618 31310 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:35.253624 31323 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:35.254566 31323 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:35.256175 31323 catalog_manager.cc:1347] Generated new cluster ID: 950b6c57b2524d44b9e56cdbe6bfc5a6
I20260809 16:38:35.256232 31323 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:35.260581 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.253422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42890 (local address 127.31.15.254:40353)
0809 16:38:35.253656 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.253666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.253889 (+ 223us) server_negotiation.cc:407] Connection header received
0809 16:38:35.254023 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.254031 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.254110 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.254262 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:35.254489 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.255861 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.256947 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.257601 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.257720 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.259778 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:35.259782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:35.259782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:35.259978 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:35.260030 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:35.260044 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:35.260104 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.260198 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.260272 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260809 16:38:35.266357 31323 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:35.266820 31323 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:35.280678 31323 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: Generated new TSK 0
I20260809 16:38:35.280853 31323 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:35.284298 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.193:0
--local_ip_for_outbound_sockets=127.31.15.193
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--builtin_ntp_servers=127.31.15.212:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 16:38:35.380949 31344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:35.381166 31344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:35.381198 31344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:35.382308 31344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:35.382350 31344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:35.382383 31344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.193
I20260809 16:38:35.384117 31344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.31.15.193
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.193
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:35.384353 31344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:35.384647 31344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:35.389354 31351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.389392 31355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.389627 31360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:35.389917 31344 server_base.cc:1034] running on GCE node
I20260809 16:38:35.390126 31344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:35.390349 31344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:35.391484 31344 hybrid_clock.cc:648] HybridClock initialized: now 1786293515391460 us; error 43 us; skew 500 ppm
I20260809 16:38:35.392585 31344 webserver.cc:460] Webserver started at http://127.31.15.193:42041/ using document root <none> and password file <none>
I20260809 16:38:35.392799 31344 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:35.392848 31344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:35.392976 31344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:35.393988 31344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data/instance:
uuid: "435934ae5c2a42948117b2683d68b228"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "3c3b30a812a9f817b0e461554fc23d34"
server_key_iv: "1d2fe6ac962a35cfb59bc8ff7fee4910"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.394330 31344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance:
uuid: "435934ae5c2a42948117b2683d68b228"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "3c3b30a812a9f817b0e461554fc23d34"
server_key_iv: "1d2fe6ac962a35cfb59bc8ff7fee4910"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.395576 31344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:35.396234 31370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.396369 31344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:35.396457 31344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal
uuid: "435934ae5c2a42948117b2683d68b228"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "3c3b30a812a9f817b0e461554fc23d34"
server_key_iv: "1d2fe6ac962a35cfb59bc8ff7fee4910"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.396520 31344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:35.408977 31344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:35.409255 31344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:35.409374 31344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:35.409574 31344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:35.409864 31344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:35.409898 31344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.409924 31344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:35.409941 31344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.415609 31344 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.193:39179
I20260809 16:38:35.415673 31550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.193:39179 every 8 connection(s)
I20260809 16:38:35.415947 31344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/data/info.pb
I20260809 16:38:35.420339 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31344
I20260809 16:38:35.420457 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-0/wal/instance
I20260809 16:38:35.420648 31807 external_mini_cluster.cc:1428] Setting key 16111a823883d23d9ace4b7f65e8171e
I20260809 16:38:35.420692 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.416589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.193:33935 (local address 127.31.15.254:40353)
0809 16:38:35.416643 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.416645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.416848 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:38:35.417306 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.417307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.417317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.417368 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:35.417994 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.418488 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.419429 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.419620 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.419671 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.420409 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:35.420411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:35.420411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:35.420416 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:35.420432 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:35.420436 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:35.420477 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.420581 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.420614 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 16:38:35.420882 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.416560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40353 (local address 127.31.15.193:33935)
0809 16:38:35.416737 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.416777 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.417235 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.417433 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.417436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.417444 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:35.417926 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.417930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.418554 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.418554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.419354 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.419357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.419678 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.419679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.419778 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.420260 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:35.420261 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:35.420289 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:35.420310 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:35.420503 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:35.420510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:35.420513 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.420575 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.420674 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260809 16:38:35.421486 31551 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40353
I20260809 16:38:35.421572 31551 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:35.421587 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.194:0
--local_ip_for_outbound_sockets=127.31.15.194
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--builtin_ntp_servers=127.31.15.212:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:38:35.421768 31551 heartbeater.cc:511] Master 127.31.15.254:40353 requested a full tablet report, sending...
I20260809 16:38:35.422269 31244 ts_manager.cc:194] Registered new tserver with Master: 435934ae5c2a42948117b2683d68b228 (127.31.15.193:39179)
I20260809 16:38:35.422860 31244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.193:33935
W20260809 16:38:35.502580 31554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:35.502772 31554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:35.502792 31554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:35.503937 31554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:35.503984 31554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:35.504009 31554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.194
I20260809 16:38:35.505299 31554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.31.15.194
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.194
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:35.505506 31554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:35.505767 31554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:35.510314 31565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.510354 31563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.510316 31571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:35.510568 31554 server_base.cc:1034] running on GCE node
I20260809 16:38:35.510761 31554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:35.510977 31554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:35.512105 31554 hybrid_clock.cc:648] HybridClock initialized: now 1786293515512085 us; error 36 us; skew 500 ppm
I20260809 16:38:35.513190 31554 webserver.cc:460] Webserver started at http://127.31.15.194:42543/ using document root <none> and password file <none>
I20260809 16:38:35.513391 31554 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:35.513434 31554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:35.513561 31554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:35.514518 31554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data/instance:
uuid: "cd847c928c4d42e29143d31cde6bf446"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "29bf2b91bec6f522d89f55c9af47dffa"
server_key_iv: "ff91c87cc1d640cafab926d0e0eccd7f"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.514827 31554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance:
uuid: "cd847c928c4d42e29143d31cde6bf446"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "29bf2b91bec6f522d89f55c9af47dffa"
server_key_iv: "ff91c87cc1d640cafab926d0e0eccd7f"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.516034 31554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:35.516693 31587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.516821 31554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:35.516893 31554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal
uuid: "cd847c928c4d42e29143d31cde6bf446"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "29bf2b91bec6f522d89f55c9af47dffa"
server_key_iv: "ff91c87cc1d640cafab926d0e0eccd7f"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.516958 31554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:35.537262 31554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:35.537534 31554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:35.537650 31554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:35.537866 31554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:35.538187 31554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:35.538221 31554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.538249 31554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:35.538266 31554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.543615 31554 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.194:42691
I20260809 16:38:35.543659 31772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.194:42691 every 8 connection(s)
I20260809 16:38:35.543955 31554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/data/info.pb
I20260809 16:38:35.546936 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31554
I20260809 16:38:35.547044 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal/instance
I20260809 16:38:35.547243 31807 external_mini_cluster.cc:1428] Setting key 039501bb94ecdf08f2b57fe3856df5d0
I20260809 16:38:35.548192 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.31.15.195:0
--local_ip_for_outbound_sockets=127.31.15.195
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--builtin_ntp_servers=127.31.15.212:39993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 16:38:35.549291 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.544649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.194:59507 (local address 127.31.15.254:40353)
0809 16:38:35.544706 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.544707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.544924 (+ 217us) server_negotiation.cc:407] Connection header received
0809 16:38:35.545431 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.545433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.545446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.545499 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:35.546150 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.546645 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.547960 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.548164 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.548221 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.549022 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:35.549023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:35.549023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:35.549028 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:35.549043 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:35.549048 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:35.549093 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.549182 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.549212 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 16:38:35.549490 31780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.544637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40353 (local address 127.31.15.194:59507)
0809 16:38:35.544805 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.544839 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.545313 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.545575 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.545578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.545586 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:35.546059 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.546064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.546732 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.546733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.547878 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.547881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.548239 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.548240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.548390 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.548891 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:35.548891 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:35.548919 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:35.548942 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:35.549098 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:35.549104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:35.549107 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.549180 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.549284 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 16:38:35.550014 31773 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40353
I20260809 16:38:35.550107 31773 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:35.550276 31773 heartbeater.cc:511] Master 127.31.15.254:40353 requested a full tablet report, sending...
I20260809 16:38:35.550621 31244 ts_manager.cc:194] Registered new tserver with Master: cd847c928c4d42e29143d31cde6bf446 (127.31.15.194:42691)
I20260809 16:38:35.551039 31244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.194:59507
W20260809 16:38:35.629077 31781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:35.629299 31781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:35.629319 31781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:35.630401 31781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:35.630434 31781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:35.630457 31781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.31.15.195
I20260809 16:38:35.631769 31781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:39993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.31.15.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.31.15.195
--webserver_port=0
--tserver_master_addrs=127.31.15.254:40353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.31.15.195
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:35.631953 31781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:35.632194 31781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:35.636835 31789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:35.636840 31790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:35.636966 31781 server_base.cc:1034] running on GCE node
W20260809 16:38:35.636840 31792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:35.637245 31781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:35.637470 31781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:35.638620 31781 hybrid_clock.cc:648] HybridClock initialized: now 1786293515638608 us; error 26 us; skew 500 ppm
I20260809 16:38:35.639914 31781 webserver.cc:460] Webserver started at http://127.31.15.195:44909/ using document root <none> and password file <none>
I20260809 16:38:35.640126 31781 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:35.640173 31781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:35.640300 31781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:35.641542 31781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data/instance:
uuid: "4f55c53f35e7414096a86683c47c4991"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "a202e49b9f022c51f4691d45ed6e8dfe"
server_key_iv: "5d7d3d4db3e47ee3b0b43f2bdbaf3ef9"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.641914 31781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance:
uuid: "4f55c53f35e7414096a86683c47c4991"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "a202e49b9f022c51f4691d45ed6e8dfe"
server_key_iv: "5d7d3d4db3e47ee3b0b43f2bdbaf3ef9"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.643144 31781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:35.643922 31803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.644059 31781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:35.644143 31781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal
uuid: "4f55c53f35e7414096a86683c47c4991"
format_stamp: "Formatted at 2026-08-09 16:38:35 on dist-test-slave-21b9"
server_key: "a202e49b9f022c51f4691d45ed6e8dfe"
server_key_iv: "5d7d3d4db3e47ee3b0b43f2bdbaf3ef9"
server_key_version: "encryptionkey@0"
I20260809 16:38:35.644207 31781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:35.661509 31781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:35.661800 31781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:35.661926 31781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:35.662142 31781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 16:38:35.662451 31781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 16:38:35.662488 31781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.662518 31781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 16:38:35.662539 31781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:35.667879 31781 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.195:38501
I20260809 16:38:35.667938 31992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.195:38501 every 8 connection(s)
I20260809 16:38:35.668248 31781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/data/info.pb
I20260809 16:38:35.672928 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.668927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:39907 (local address 127.31.15.254:40353)
0809 16:38:35.668985 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.668987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.669208 (+ 221us) server_negotiation.cc:407] Connection header received
0809 16:38:35.669663 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.669664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.669674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.669731 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:35.670330 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.670820 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.671758 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.671947 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.671999 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.672745 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:35.672746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:35.672746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:35.672749 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:35.672760 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:35.672763 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:35.672791 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.672848 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.672868 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:38:35.673110 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.668903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:40353 (local address 127.31.15.195:39907)
0809 16:38:35.669079 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.669119 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.669590 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.669783 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.669786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.669794 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:38:35.670269 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.670273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.670882 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.670883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.671693 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.671696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.672005 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.672006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.672105 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.672631 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 16:38:35.672632 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:38:35.672660 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 16:38:35.672683 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:38:35.672797 (+ 114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:38:35.672800 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:38:35.672802 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.672844 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.672920 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260809 16:38:35.673532 31993 heartbeater.cc:347] Connected to a master server at 127.31.15.254:40353
I20260809 16:38:35.673621 31993 heartbeater.cc:464] Registering TS with master...
I20260809 16:38:35.673794 31993 heartbeater.cc:511] Master 127.31.15.254:40353 requested a full tablet report, sending...
I20260809 16:38:35.674026 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 31781
I20260809 16:38:35.674146 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal/instance
I20260809 16:38:35.674206 31244 ts_manager.cc:194] Registered new tserver with Master: 4f55c53f35e7414096a86683c47c4991 (127.31.15.195:38501)
I20260809 16:38:35.674340 31807 external_mini_cluster.cc:1428] Setting key 8828ceb1b528067bde43376fc744a7d4
I20260809 16:38:35.674683 31244 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.31.15.195:39907
I20260809 16:38:35.675208 31807 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 16:38:35.679054 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.676504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.31.15.254:40353)
0809 16:38:35.676569 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.676570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.676625 (+ 55us) server_negotiation.cc:407] Connection header received
0809 16:38:35.676697 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.676698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.676709 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.676746 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:35.676906 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.677354 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.678053 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.678247 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.678299 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.678843 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:35.678844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:35.678844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:35.678848 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:35.678861 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:35.678866 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:35.678909 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.678961 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.678985 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 16:38:35.680528 31244 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 16:38:35.682009 31244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 16:38:35.688081 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:39179 (local address 127.0.0.1:52292)
0809 16:38:35.684468 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.684478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.684534 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.684705 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.684706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.684707 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.684802 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.684804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.685794 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.685795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.686879 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.686882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.687839 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.687840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.687895 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.687905 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.687963 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.688010 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":22,"thread_start_us":78,"threads_started":1}
I20260809 16:38:35.688124 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.31.15.193:39179)
0809 16:38:35.684329 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.684334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.684534 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:38:35.684622 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.684626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.684641 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.684707 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.684965 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.685713 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.686978 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.687681 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.687744 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.687810 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.688004 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.688046 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260809 16:38:35.688375 32012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.195:38501 (local address 127.0.0.1:45688)
0809 16:38:35.684371 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.684402 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.684475 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.684698 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.684701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.684703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.684802 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.684805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.686274 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.686275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.687369 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.687371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.688180 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.688181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.688237 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.688241 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.688266 (+ 25us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.688302 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":21,"thread_start_us":62,"threads_started":1}
I20260809 16:38:35.688514 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:42691 (local address 127.0.0.1:55920)
0809 16:38:35.684798 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.684806 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.684857 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.685011 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.685012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.685013 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.685115 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.685117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.686318 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.686319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.687141 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.687143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.688142 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.688143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.688363 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.688367 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.688412 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.688449 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260809 16:38:35.688763 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.31.15.194:42691)
0809 16:38:35.684639 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.684643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.684873 (+ 230us) server_negotiation.cc:407] Connection header received
0809 16:38:35.684925 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.684928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.684943 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.685006 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.685189 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.686255 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.687211 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.687947 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.688017 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.688102 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.688642 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.688682 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260809 16:38:35.688763 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.684137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45688 (local address 127.31.15.195:38501)
0809 16:38:35.684423 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.684426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.684460 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:38:35.684586 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.684589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.684604 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.684677 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.685077 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.686185 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.687447 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.688095 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.688543 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.688639 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.688661 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.688692 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260809 16:38:35.688968 31444 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad4144f14a46778cede1c57d302f07 (DEFAULT_TABLE table=test-table [id=a7404d082fc240f78b37d2302050853d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:35.689352 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad4144f14a46778cede1c57d302f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:35.689397 31887 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad4144f14a46778cede1c57d302f07 (DEFAULT_TABLE table=test-table [id=a7404d082fc240f78b37d2302050853d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:35.689388 31658 tablet_service.cc:1467] Processing CreateTablet for tablet 62ad4144f14a46778cede1c57d302f07 (DEFAULT_TABLE table=test-table [id=a7404d082fc240f78b37d2302050853d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 16:38:35.689739 31887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad4144f14a46778cede1c57d302f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:35.689739 31658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ad4144f14a46778cede1c57d302f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:35.691967 32016 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: Bootstrap starting.
I20260809 16:38:35.692588 32016 tablet_bootstrap.cc:654] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:35.692876 32016 log.cc:824] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:35.693028 32022 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Bootstrap starting.
I20260809 16:38:35.693452 32024 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Bootstrap starting.
I20260809 16:38:35.693603 32016 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: No bootstrap required, opened a new log
I20260809 16:38:35.693676 32016 ts_tablet_manager.cc:1397] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:35.693711 32022 tablet_bootstrap.cc:654] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:35.694037 32022 log.cc:824] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:35.694074 32024 tablet_bootstrap.cc:654] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:35.694375 32024 log.cc:824] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:35.694821 32022 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: No bootstrap required, opened a new log
I20260809 16:38:35.694893 32022 ts_tablet_manager.cc:1397] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:35.695115 32024 tablet_bootstrap.cc:492] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: No bootstrap required, opened a new log
I20260809 16:38:35.695182 32024 ts_tablet_manager.cc:1397] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 16:38:35.695960 32016 raft_consensus.cc:348] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.696090 32016 raft_consensus.cc:374] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:35.696123 32016 raft_consensus.cc:729] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 435934ae5c2a42948117b2683d68b228, State: Initialized, Role: FOLLOWER
I20260809 16:38:35.696256 32016 consensus_queue.cc:260] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.696489 32016 ts_tablet_manager.cc:1428] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:35.696615 31551 heartbeater.cc:503] Master 127.31.15.254:40353 was elected leader, sending a full tablet report...
I20260809 16:38:35.697321 32022 raft_consensus.cc:348] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.697480 32022 raft_consensus.cc:374] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:35.697539 32024 raft_consensus.cc:348] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.697682 32024 raft_consensus.cc:374] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:35.697526 32022 raft_consensus.cc:729] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f55c53f35e7414096a86683c47c4991, State: Initialized, Role: FOLLOWER
I20260809 16:38:35.697731 32024 raft_consensus.cc:729] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd847c928c4d42e29143d31cde6bf446, State: Initialized, Role: FOLLOWER
I20260809 16:38:35.697829 32022 consensus_queue.cc:260] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.697875 32024 consensus_queue.cc:260] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.698102 32024 ts_tablet_manager.cc:1428] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 16:38:35.698102 32022 ts_tablet_manager.cc:1428] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 16:38:35.698153 31773 heartbeater.cc:503] Master 127.31.15.254:40353 was elected leader, sending a full tablet report...
I20260809 16:38:35.698153 31993 heartbeater.cc:503] Master 127.31.15.254:40353 was elected leader, sending a full tablet report...
I20260809 16:38:35.788348 32030 raft_consensus.cc:482] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 16:38:35.788486 32030 raft_consensus.cc:504] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.788792 32030 leader_election.cc:288] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 435934ae5c2a42948117b2683d68b228 (127.31.15.193:39179), cd847c928c4d42e29143d31cde6bf446 (127.31.15.194:42691)
I20260809 16:38:35.791738 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.789077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:47501 (local address 127.31.15.193:39179)
0809 16:38:35.789138 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.789140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.789152 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:38:35.789186 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.789187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.789199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.789267 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.789437 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.789981 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791001 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.791410 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791465 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.791472 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.791631 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.791671 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 16:38:35.791738 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.789004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:39179 (local address 127.31.15.195:47501)
0809 16:38:35.789056 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.789070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.789115 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.789285 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.789286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.789288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.789361 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.789362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.790056 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.790057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.790907 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.790910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.791481 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.791567 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.791571 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.791618 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.791653 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":33}
I20260809 16:38:35.791776 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.789208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.31.15.195:49695 (local address 127.31.15.194:42691)
0809 16:38:35.789271 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.789273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.789361 (+ 88us) server_negotiation.cc:407] Connection header received
0809 16:38:35.789413 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.789414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.789424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.789473 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 16:38:35.789645 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.790159 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791059 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.791466 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791538 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.791547 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.791678 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.791718 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 16:38:35.791829 32032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.789122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.194:42691 (local address 127.31.15.195:49695)
0809 16:38:35.789300 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.789311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:38:35.789363 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:38:35.789507 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:38:35.789508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:38:35.789510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 16:38:35.789580 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.789582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.790229 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.790229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.790978 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:38:35.790980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.791545 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.791546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:38:35.791607 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.791611 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 16:38:35.791665 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 16:38:35.791697 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260809 16:38:35.792168 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad4144f14a46778cede1c57d302f07" candidate_uuid: "4f55c53f35e7414096a86683c47c4991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd847c928c4d42e29143d31cde6bf446" is_pre_election: true
I20260809 16:38:35.792171 31477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad4144f14a46778cede1c57d302f07" candidate_uuid: "4f55c53f35e7414096a86683c47c4991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435934ae5c2a42948117b2683d68b228" is_pre_election: true
I20260809 16:38:35.792327 31477 raft_consensus.cc:2454] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f55c53f35e7414096a86683c47c4991 in term 0.
I20260809 16:38:35.792327 31696 raft_consensus.cc:2454] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f55c53f35e7414096a86683c47c4991 in term 0.
I20260809 16:38:35.792562 31821 leader_election.cc:302] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f55c53f35e7414096a86683c47c4991, cd847c928c4d42e29143d31cde6bf446; no voters:
I20260809 16:38:35.792690 32030 raft_consensus.cc:2781] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 16:38:35.792747 32030 raft_consensus.cc:482] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 16:38:35.792761 32030 raft_consensus.cc:3037] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.793433 32030 raft_consensus.cc:504] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.793542 32030 leader_election.cc:288] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [CANDIDATE]: Term 1 election: Requested vote from peers 435934ae5c2a42948117b2683d68b228 (127.31.15.193:39179), cd847c928c4d42e29143d31cde6bf446 (127.31.15.194:42691)
I20260809 16:38:35.793712 31477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad4144f14a46778cede1c57d302f07" candidate_uuid: "4f55c53f35e7414096a86683c47c4991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "435934ae5c2a42948117b2683d68b228"
I20260809 16:38:35.793730 31696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ad4144f14a46778cede1c57d302f07" candidate_uuid: "4f55c53f35e7414096a86683c47c4991" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd847c928c4d42e29143d31cde6bf446"
I20260809 16:38:35.793797 31696 raft_consensus.cc:3037] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:35.793797 31477 raft_consensus.cc:3037] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:38:35.794435 31777 tablet.cc:2369] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 16:38:35.794710 31696 raft_consensus.cc:2454] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f55c53f35e7414096a86683c47c4991 in term 1.
I20260809 16:38:35.794710 31477 raft_consensus.cc:2454] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f55c53f35e7414096a86683c47c4991 in term 1.
I20260809 16:38:35.794874 31821 leader_election.cc:302] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 435934ae5c2a42948117b2683d68b228, 4f55c53f35e7414096a86683c47c4991; no voters:
I20260809 16:38:35.794977 32030 raft_consensus.cc:2781] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:35.795120 32030 raft_consensus.cc:686] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 1 LEADER]: Becoming Leader. State: Replica: 4f55c53f35e7414096a86683c47c4991, State: Running, Role: LEADER
I20260809 16:38:35.795202 32030 consensus_queue.cc:237] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } }
I20260809 16:38:35.796057 31244 catalog_manager.cc:5495] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f55c53f35e7414096a86683c47c4991 (127.31.15.195). New cstate: current_term: 1 leader_uuid: "4f55c53f35e7414096a86683c47c4991" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f55c53f35e7414096a86683c47c4991" member_type: VOTER last_known_addr { host: "127.31.15.195" port: 38501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 } health_report { overall_health: UNKNOWN } } }
I20260809 16:38:35.803620 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.801610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.31.15.195:38501)
0809 16:38:35.801662 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:35.801663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:35.801671 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:38:35.801719 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:35.801720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:35.801734 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:35.801790 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:38:35.801880 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.802307 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.802979 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:35.803152 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:35.803192 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:35.803331 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:38:35.803420 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:38:35.803479 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:35.803523 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:38:35.803537 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:38:35.805166 31477 raft_consensus.cc:1261] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228 [term 1 FOLLOWER]: Refusing update from remote peer 4f55c53f35e7414096a86683c47c4991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:35.805166 31696 raft_consensus.cc:1261] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 1 FOLLOWER]: Refusing update from remote peer 4f55c53f35e7414096a86683c47c4991: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:38:35.805459 32030 consensus_queue.cc:1035] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "435934ae5c2a42948117b2683d68b228" member_type: VOTER last_known_addr { host: "127.31.15.193" port: 39179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:35.805586 32034 consensus_queue.cc:1035] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd847c928c4d42e29143d31cde6bf446" member_type: VOTER last_known_addr { host: "127.31.15.194" port: 42691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:38:35.809921 31244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:35.810025 31244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 16:38:35.810711 31244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b1dc78a2f9b1426b8805a570ccca8549: Sending DeleteTablet for 3 replicas of tablet 62ad4144f14a46778cede1c57d302f07
I20260809 16:38:35.811003 31887 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad4144f14a46778cede1c57d302f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='slave'} at 127.0.0.1:45688
I20260809 16:38:35.811048 31658 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad4144f14a46778cede1c57d302f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='slave'} at 127.0.0.1:55920
I20260809 16:38:35.811081 31444 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ad4144f14a46778cede1c57d302f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 16:38:35 UTC) from {username='slave'} at 127.0.0.1:52292
I20260809 16:38:35.811293 32055 tablet_replica.cc:330] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: stopping tablet replica
I20260809 16:38:35.811352 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31344
I20260809 16:38:35.811391 32055 raft_consensus.cc:2229] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 1 LEADER]: Raft consensus shutting down.
I20260809 16:38:35.811475 32055 raft_consensus.cc:2258] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:35.811575 32059 tablet_replica.cc:330] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: stopping tablet replica
I20260809 16:38:35.811643 32062 tablet_replica.cc:330] T 62ad4144f14a46778cede1c57d302f07 P 435934ae5c2a42948117b2683d68b228: stopping tablet replica
I20260809 16:38:35.811761 32059 raft_consensus.cc:2229] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 16:38:35.811968 32059 raft_consensus.cc:2258] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 16:38:35.817307 32059 ts_tablet_manager.cc:1905] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:35.819448 32059 ts_tablet_manager.cc:1918] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:35.819622 32059 log.cc:1196] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-1/wal/wals/62ad4144f14a46778cede1c57d302f07
I20260809 16:38:35.820077 32059 ts_tablet_manager.cc:1939] T 62ad4144f14a46778cede1c57d302f07 P cd847c928c4d42e29143d31cde6bf446: Deleting consensus metadata
I20260809 16:38:35.820761 31221 catalog_manager.cc:4841] TS cd847c928c4d42e29143d31cde6bf446 (127.31.15.194:42691): tablet 62ad4144f14a46778cede1c57d302f07 (table test-table [id=a7404d082fc240f78b37d2302050853d]) successfully deleted
I20260809 16:38:35.829916 32055 ts_tablet_manager.cc:1905] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 16:38:35.831583 32055 ts_tablet_manager.cc:1918] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 16:38:35.831667 32055 log.cc:1196] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Deleting WAL directory at /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786293272397626-31807-0/minicluster-data/ts-2/wal/wals/62ad4144f14a46778cede1c57d302f07
I20260809 16:38:35.831907 32055 ts_tablet_manager.cc:1939] T 62ad4144f14a46778cede1c57d302f07 P 4f55c53f35e7414096a86683c47c4991: Deleting consensus metadata
I20260809 16:38:35.832362 31220 catalog_manager.cc:4841] TS 4f55c53f35e7414096a86683c47c4991 (127.31.15.195:38501): tablet 62ad4144f14a46778cede1c57d302f07 (table test-table [id=a7404d082fc240f78b37d2302050853d]) successfully deleted
W20260809 16:38:35.833875 31221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.31.15.193:39179 (error 108)
I20260809 16:38:35.834100 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31554
I20260809 16:38:35.844396 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:35.844237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.193:39179 (local address 127.0.0.1:52308)
0809 16:38:35.844298 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:38:35.844353 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.193:39179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 16:38:35.844560 31221 catalog_manager.cc:4584] TS 435934ae5c2a42948117b2683d68b228 (127.31.15.193:39179): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62ad4144f14a46778cede1c57d302f07: Network error: Client connection negotiation failed: client connection to 127.31.15.193:39179: connect: Connection refused (error 111)
I20260809 16:38:35.847837 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31781
I20260809 16:38:35.856556 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 31192
2026-08-09T16:38:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (758 ms)
[----------] 6 tests from EncryptionPolicyTest (8521 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:35 dist-test-slave-21b9 krb5kdc[32069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:35 dist-test-slave-21b9 krb5kdc[32069](info): set up 2 sockets
Aug 09 16:38:35 dist-test-slave-21b9 krb5kdc[32069](info): commencing operation
krb5kdc: starting...
W20260809 16:38:37.012209 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:37.125934 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:37.209370 32087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:37.209597 32087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:37.209632 32087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:37.210788 32087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:37.210837 32087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:37.210861 32087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:37.210880 32087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:37.210899 32087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:37.212443 32087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34951
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:37.212699 32087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:37.212918 32087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:37.217679 32094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:37.217648 32099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:37.217646 32093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:37.217664 32087 server_base.cc:1034] running on GCE node
I20260809 16:38:37.218117 32087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:37.218458 32087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:37.219606 32087 hybrid_clock.cc:648] HybridClock initialized: now 1786293517219586 us; error 34 us; skew 500 ppm
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293517, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:37.222028 32087 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:37.222520 32087 webserver.cc:460] Webserver started at http://127.0.0.1:41731/ using document root <none> and password file <none>
I20260809 16:38:37.222798 32087 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:37.222851 32087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:37.222991 32087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:37.224352 32087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "0e07c78851e64ac38c790ff083185436"
format_stamp: "Formatted at 2026-08-09 16:38:37 on dist-test-slave-21b9"
server_key: "8aa2c904737836fa4fab3c934e9d51f1"
server_key_iv: "8102fe9ec1c5a5cec35b8be4e928bb7d"
server_key_version: "encryptionkey@0"
I20260809 16:38:37.224787 32087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "0e07c78851e64ac38c790ff083185436"
format_stamp: "Formatted at 2026-08-09 16:38:37 on dist-test-slave-21b9"
server_key: "8aa2c904737836fa4fab3c934e9d51f1"
server_key_iv: "8102fe9ec1c5a5cec35b8be4e928bb7d"
server_key_version: "encryptionkey@0"
I20260809 16:38:37.226438 32087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:37.227258 32108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:37.227411 32087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:37.227492 32087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "0e07c78851e64ac38c790ff083185436"
format_stamp: "Formatted at 2026-08-09 16:38:37 on dist-test-slave-21b9"
server_key: "8aa2c904737836fa4fab3c934e9d51f1"
server_key_iv: "8102fe9ec1c5a5cec35b8be4e928bb7d"
server_key_version: "encryptionkey@0"
I20260809 16:38:37.227550 32087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:37.244202 32087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:37.245841 32087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:37.245949 32087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:37.249854 32087 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34951
I20260809 16:38:37.249888 32190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34951 every 8 connection(s)
I20260809 16:38:37.250203 32087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:37.250561 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32087
I20260809 16:38:37.250690 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:37.250910 31807 external_mini_cluster.cc:1428] Setting key a088e32e59521cd0658116b964b77bdb
I20260809 16:38:37.251204 32192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:37.253897 32192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436: Bootstrap starting.
I20260809 16:38:37.254583 32192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:37.254895 32192 log.cc:824] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:37.255669 32192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436: No bootstrap required, opened a new log
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:38:37.257997 32192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 16:38:37.258160 32192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:37.258210 32192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e07c78851e64ac38c790ff083185436, State: Initialized, Role: FOLLOWER
I20260809 16:38:37.258337 32192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 16:38:37.258410 32192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:37.258440 32192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:37.258484 32192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:37.259155 32192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 16:38:37.259259 32192 leader_election.cc:302] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e07c78851e64ac38c790ff083185436; no voters:
I20260809 16:38:37.259441 32192 leader_election.cc:288] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:37.259500 32200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:37.259647 32200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [term 1 LEADER]: Becoming Leader. State: Replica: 0e07c78851e64ac38c790ff083185436, State: Running, Role: LEADER
I20260809 16:38:37.259702 32192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:37.259737 32200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } }
I20260809 16:38:37.260126 32203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e07c78851e64ac38c790ff083185436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260809 16:38:37.260227 32203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:37.260385 32204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e07c78851e64ac38c790ff083185436. Latest consensus state: current_term: 1 leader_uuid: "0e07c78851e64ac38c790ff083185436" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e07c78851e64ac38c790ff083185436" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34951 } } }
I20260809 16:38:37.260463 32204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:37.261133 32210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:37.261664 32210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:37.262121 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:37.251411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53215 (local address 127.0.0.1:34951)
0809 16:38:37.251597 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:37.251607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:37.251645 (+ 38us) server_negotiation.cc:407] Connection header received
0809 16:38:37.252254 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:37.252260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:37.252306 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:37.252426 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:37.253066 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:37.253912 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:37.254916 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:37.255471 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:37.255562 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:37.258119 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:37.258123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:37.258123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:37.258133 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:37.259952 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:37.260561 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:37.260565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:37.260565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:37.260667 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:37.260924 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:37.260925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:37.260926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:37.261277 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:37.261407 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:37.261810 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 16:38:37.261912 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 16:38:37.263095 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:37.264321 32210 catalog_manager.cc:1347] Generated new cluster ID: 4dc804bdf1e945388b15b439dd57f65f
I20260809 16:38:37.264374 32210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:37.274210 32210 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:37.274825 32210 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:37.275400 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:37.264580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50029 (local address 127.0.0.1:34951)
0809 16:38:37.264611 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:37.264613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:37.264623 (+ 10us) server_negotiation.cc:407] Connection header received
0809 16:38:37.264657 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:37.264657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:37.264665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:37.264706 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:37.265317 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:37.266021 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:37.267022 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:37.267375 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:37.267427 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:37.272476 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:37.272477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:37.272477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:37.272484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:37.274015 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:37.274480 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:37.274482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:37.274482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:37.274551 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:37.274770 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:37.274771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:37.274771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:37.274884 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:37.274984 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:37.275205 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 16:38:37.275309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 16:38:37.283981 32210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e07c78851e64ac38c790ff083185436: Generated new TSK 0
I20260809 16:38:37.284147 32210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:37.287168 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32087
W20260809 16:38:37.295487 32064 connection.cc:535] client connection to 127.0.0.1:34951 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34951): Connection reset by peer (error 104)
2026-08-09T16:38:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32237](info): set up 2 sockets
Aug 09 16:38:37 dist-test-slave-21b9 krb5kdc[32237](info): commencing operation
krb5kdc: starting...
W20260809 16:38:38.431794 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:38 dist-test-slave-21b9 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:38.545389 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38481
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:38.625849 32263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:38.626039 32263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:38.626060 32263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:38.627178 32263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:38.627219 32263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:38.627233 32263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:38.627243 32263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:38.627254 32263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:38.628758 32263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38481
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:38.628984 32263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:38.629217 32263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:38.633818 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:38.633792 32279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:38.633834 32273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:38.633982 32263 server_base.cc:1034] running on GCE node
I20260809 16:38:38.634198 32263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:38.634464 32263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:38.635602 32263 hybrid_clock.cc:648] HybridClock initialized: now 1786293518635589 us; error 25 us; skew 500 ppm
Aug 09 16:38:38 dist-test-slave-21b9 krb5kdc[32237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293518, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:38.637956 32263 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:38.638382 32263 webserver.cc:460] Webserver started at http://127.0.0.1:39001/ using document root <none> and password file <none>
I20260809 16:38:38.638607 32263 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:38.638655 32263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:38.638779 32263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:38.639861 32263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "c1f137e02e184e65873bf28ef33a8381"
format_stamp: "Formatted at 2026-08-09 16:38:38 on dist-test-slave-21b9"
server_key: "8154491126491bde2068d82a3bd02ea7"
server_key_iv: "a10ee69e3a739ec31c3e02fe97ed0456"
server_key_version: "encryptionkey@0"
I20260809 16:38:38.640184 32263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "c1f137e02e184e65873bf28ef33a8381"
format_stamp: "Formatted at 2026-08-09 16:38:38 on dist-test-slave-21b9"
server_key: "8154491126491bde2068d82a3bd02ea7"
server_key_iv: "a10ee69e3a739ec31c3e02fe97ed0456"
server_key_version: "encryptionkey@0"
I20260809 16:38:38.641408 32263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:38.642141 32289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:38.642288 32263 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:38.642362 32263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "c1f137e02e184e65873bf28ef33a8381"
format_stamp: "Formatted at 2026-08-09 16:38:38 on dist-test-slave-21b9"
server_key: "8154491126491bde2068d82a3bd02ea7"
server_key_iv: "a10ee69e3a739ec31c3e02fe97ed0456"
server_key_version: "encryptionkey@0"
I20260809 16:38:38.642426 32263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:38.652678 32263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:38.654656 32263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:38.654788 32263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:38.658614 32263 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38481
I20260809 16:38:38.658653 32377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38481 every 8 connection(s)
I20260809 16:38:38.658963 32263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:38.659677 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32263
I20260809 16:38:38.659791 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:38.659893 32378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:38.659978 31807 external_mini_cluster.cc:1428] Setting key ab7e633b0c6331f40a42f20011fa048d
I20260809 16:38:38.663223 32378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381: Bootstrap starting.
I20260809 16:38:38.663867 32378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:38.664144 32378 log.cc:824] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:38.664808 32378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381: No bootstrap required, opened a new log
I20260809 16:38:38.665741 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:38.660465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46959 (local address 127.0.0.1:38481)
0809 16:38:38.660655 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:38.660665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:38.660699 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:38:38.661402 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:38.661412 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:38.661477 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:38.661638 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:38.661862 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:38.663179 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:38.663992 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:38.664408 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:38.664525 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:38.665185 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:38.665189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:38.665190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:38.665339 (+ 149us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:38.665371 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:38.665381 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:38.665450 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:38.665546 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:38:38.665573 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260809 16:38:38.666420 32378 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } }
I20260809 16:38:38.666589 32378 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:38.666625 32378 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1f137e02e184e65873bf28ef33a8381, State: Initialized, Role: FOLLOWER
I20260809 16:38:38.666739 32378 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } }
I20260809 16:38:38.666815 32378 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:38.666854 32378 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:38.666900 32378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:38.666989 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:38.667491 32378 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } }
I20260809 16:38:38.667590 32378 leader_election.cc:302] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1f137e02e184e65873bf28ef33a8381; no voters:
I20260809 16:38:38.667742 32378 leader_election.cc:288] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:38.667773 32384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:38.667886 32384 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [term 1 LEADER]: Becoming Leader. State: Replica: c1f137e02e184e65873bf28ef33a8381, State: Running, Role: LEADER
I20260809 16:38:38.667989 32378 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:38.668017 32384 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } }
I20260809 16:38:38.668361 32384 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1f137e02e184e65873bf28ef33a8381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } } }
I20260809 16:38:38.668506 32384 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:38.668644 32384 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1f137e02e184e65873bf28ef33a8381. Latest consensus state: current_term: 1 leader_uuid: "c1f137e02e184e65873bf28ef33a8381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1f137e02e184e65873bf28ef33a8381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38481 } } }
I20260809 16:38:38.668715 32384 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:38.668799 32406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:38.669346 32406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:38.670892 32406 catalog_manager.cc:1347] Generated new cluster ID: b2f424d326cf4ab2a8814175d1c3b9d0
I20260809 16:38:38.670935 32406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:38.671728 32382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:38.668276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48553 (local address 127.0.0.1:38481)
0809 16:38:38.668310 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:38.668311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:38.668429 (+ 118us) server_negotiation.cc:407] Connection header received
0809 16:38:38.668479 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:38.668480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:38.668489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:38.668541 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:38.668722 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:38.669415 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:38.670429 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:38.670719 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:38.670785 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:38.671506 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:38.671507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:38.671508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:38.671512 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:38.671526 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:38.671530 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:38.671584 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:38.671657 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 16:38:38.671667 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:38:38.686548 32406 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:38.687300 32406 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:38.693532 32406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1f137e02e184e65873bf28ef33a8381: Generated new TSK 0
I20260809 16:38:38.693742 32406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:38.695822 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32263
W20260809 16:38:38.714288 32232 connection.cc:535] client connection to 127.0.0.1:38481 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38481): Connection reset by peer (error 104)
2026-08-09T16:38:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:38 dist-test-slave-21b9 krb5kdc[32425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:38 dist-test-slave-21b9 krb5kdc[32425](info): set up 2 sockets
Aug 09 16:38:38 dist-test-slave-21b9 krb5kdc[32425](info): commencing operation
krb5kdc: starting...
W20260809 16:38:39.841732 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:39 dist-test-slave-21b9 krb5kdc[32425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:39.956072 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:40.041436 32453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:40.041644 32453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:40.041668 32453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:40.042737 32453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:40.042774 32453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:40.042789 32453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:40.042809 32453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:40.042824 32453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:40.044307 32453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35745
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:40.044519 32453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:40.044699 32453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:40.049211 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:40.049424 32453 server_base.cc:1034] running on GCE node
W20260809 16:38:40.049182 32461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:40.049201 32465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:40.049736 32453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:40.050015 32453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:40.051177 32453 hybrid_clock.cc:648] HybridClock initialized: now 1786293520051161 us; error 33 us; skew 500 ppm
Aug 09 16:38:40 dist-test-slave-21b9 krb5kdc[32425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293520, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:40.053609 32453 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:40.054044 32453 webserver.cc:460] Webserver started at http://127.0.0.1:34229/ using document root <none> and password file <none>
I20260809 16:38:40.054237 32453 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:40.054281 32453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:40.054410 32453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:40.055477 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "a1af9743514e4ef3adfbe0d92c1a2079"
format_stamp: "Formatted at 2026-08-09 16:38:40 on dist-test-slave-21b9"
server_key: "3a54e4405abf28000d5f3a560f1e0a91"
server_key_iv: "98f7658fdd19bfe98ea810a93f409c6c"
server_key_version: "encryptionkey@0"
I20260809 16:38:40.055804 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "a1af9743514e4ef3adfbe0d92c1a2079"
format_stamp: "Formatted at 2026-08-09 16:38:40 on dist-test-slave-21b9"
server_key: "3a54e4405abf28000d5f3a560f1e0a91"
server_key_iv: "98f7658fdd19bfe98ea810a93f409c6c"
server_key_version: "encryptionkey@0"
I20260809 16:38:40.057086 32453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:40.057780 32478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:40.057934 32453 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:40.058017 32453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "a1af9743514e4ef3adfbe0d92c1a2079"
format_stamp: "Formatted at 2026-08-09 16:38:40 on dist-test-slave-21b9"
server_key: "3a54e4405abf28000d5f3a560f1e0a91"
server_key_iv: "98f7658fdd19bfe98ea810a93f409c6c"
server_key_version: "encryptionkey@0"
I20260809 16:38:40.058081 32453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:40.061301 32453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:40.062925 32453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:40.063042 32453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:40.066800 32453 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35745
I20260809 16:38:40.066833 32548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35745 every 8 connection(s)
I20260809 16:38:40.067117 32453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:40.068079 32550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:40.070222 32550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079: Bootstrap starting.
I20260809 16:38:40.070762 32550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:40.070894 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32453
I20260809 16:38:40.071024 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:40.071101 32550 log.cc:824] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:40.071214 31807 external_mini_cluster.cc:1428] Setting key 107ece6a7095022a2775107c253420bb
I20260809 16:38:40.071868 32550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079: No bootstrap required, opened a new log
I20260809 16:38:40.073693 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:40.071715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47677 (local address 127.0.0.1:35745)
0809 16:38:40.071896 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:40.071903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:40.071935 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:38:40.072364 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:40.072370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:40.072394 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:40.072514 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:40.073172 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:40.073175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:40.073176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:40.073180 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:40.073218 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:40.073227 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:40.073298 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:40.073349 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:38:40.073386 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260809 16:38:40.073737 32550 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260809 16:38:40.073904 32550 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:40.073948 32550 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1af9743514e4ef3adfbe0d92c1a2079, State: Initialized, Role: FOLLOWER
I20260809 16:38:40.074064 32550 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260809 16:38:40.074159 32550 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:40.074303 32550 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:40.074368 32550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:40.074568 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:40.075006 32550 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260809 16:38:40.075103 32550 leader_election.cc:302] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1af9743514e4ef3adfbe0d92c1a2079; no voters:
I20260809 16:38:40.075250 32550 leader_election.cc:288] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:40.075312 32559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:40.075446 32559 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [term 1 LEADER]: Becoming Leader. State: Replica: a1af9743514e4ef3adfbe0d92c1a2079, State: Running, Role: LEADER
I20260809 16:38:40.075489 32550 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:40.075619 32559 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } }
I20260809 16:38:40.076184 32564 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1af9743514e4ef3adfbe0d92c1a2079. Latest consensus state: current_term: 1 leader_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } } }
I20260809 16:38:40.076285 32564 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:40.076463 32563 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1af9743514e4ef3adfbe0d92c1a2079" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35745 } } }
I20260809 16:38:40.076532 32563 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:40.076668 32590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:40.077186 32558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:40.075957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50261 (local address 127.0.0.1:35745)
0809 16:38:40.075992 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:40.075994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:40.076224 (+ 230us) server_negotiation.cc:407] Connection header received
0809 16:38:40.076321 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:40.076322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:40.076327 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:40.076353 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:40.076953 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:40.076955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:40.076955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:40.076959 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:40.076973 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:40.076977 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:40.077005 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:40.077120 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 16:38:40.077132 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:38:40.077189 32590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:40.078976 32590 catalog_manager.cc:1347] Generated new cluster ID: f3994c60453a4343ae816b4af08b568e
I20260809 16:38:40.079025 32590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:40.085620 32590 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:40.085803 32590 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:40.090085 32590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1af9743514e4ef3adfbe0d92c1a2079: Generated new TSK 0
I20260809 16:38:40.090231 32590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:40.098814 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32453
2026-08-09T16:38:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:40 dist-test-slave-21b9 krb5kdc[32612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:40 dist-test-slave-21b9 krb5kdc[32612](info): set up 2 sockets
Aug 09 16:38:40 dist-test-slave-21b9 krb5kdc[32612](info): commencing operation
krb5kdc: starting...
W20260809 16:38:41.253253 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[32612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:41.367650 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:41.448956 32633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:41.449183 32633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:41.449215 32633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:41.450349 32633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:41.450392 32633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:41.450410 32633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:41.450428 32633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:41.450450 32633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:41.451982 32633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:41.452232 32633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:41.452454 32633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:41.457170 32644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:41.457194 32649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:41.457178 32646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:41.457465 32633 server_base.cc:1034] running on GCE node
I20260809 16:38:41.457690 32633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:41.457954 32633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:41.459093 32633 hybrid_clock.cc:648] HybridClock initialized: now 1786293521459081 us; error 31 us; skew 500 ppm
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[32612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293521, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:41.461540 32633 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:41.461988 32633 webserver.cc:460] Webserver started at http://127.0.0.1:34445/ using document root <none> and password file <none>
I20260809 16:38:41.462216 32633 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:41.462265 32633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:41.462391 32633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:41.463523 32633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "eee3abc0d808447b89cadeba3b3d3d1a"
format_stamp: "Formatted at 2026-08-09 16:38:41 on dist-test-slave-21b9"
server_key: "b6ee99271cb23fec2296494a1f704cfa"
server_key_iv: "eb71e6b095d5cef73b346f20288be8cb"
server_key_version: "encryptionkey@0"
I20260809 16:38:41.463850 32633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "eee3abc0d808447b89cadeba3b3d3d1a"
format_stamp: "Formatted at 2026-08-09 16:38:41 on dist-test-slave-21b9"
server_key: "b6ee99271cb23fec2296494a1f704cfa"
server_key_iv: "eb71e6b095d5cef73b346f20288be8cb"
server_key_version: "encryptionkey@0"
I20260809 16:38:41.465101 32633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:41.465808 32658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:41.465965 32633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:41.466053 32633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "eee3abc0d808447b89cadeba3b3d3d1a"
format_stamp: "Formatted at 2026-08-09 16:38:41 on dist-test-slave-21b9"
server_key: "b6ee99271cb23fec2296494a1f704cfa"
server_key_iv: "eb71e6b095d5cef73b346f20288be8cb"
server_key_version: "encryptionkey@0"
I20260809 16:38:41.466113 32633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:41.476814 32633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:41.478467 32633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:41.478598 32633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:41.482582 32633 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41653
I20260809 16:38:41.482628 32741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41653 every 8 connection(s)
I20260809 16:38:41.482942 32633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:41.483783 32743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:41.485998 32743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a: Bootstrap starting.
I20260809 16:38:41.486526 32743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:41.486755 32743 log.cc:824] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:41.487390 32743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a: No bootstrap required, opened a new log
I20260809 16:38:41.488929 32743 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260809 16:38:41.489073 32743 raft_consensus.cc:374] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:41.489096 32743 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee3abc0d808447b89cadeba3b3d3d1a, State: Initialized, Role: FOLLOWER
I20260809 16:38:41.489224 32743 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260809 16:38:41.489305 32743 raft_consensus.cc:388] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:41.489331 32743 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:41.489363 32743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:41.489830 32743 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260809 16:38:41.489915 32743 leader_election.cc:302] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eee3abc0d808447b89cadeba3b3d3d1a; no voters:
I20260809 16:38:41.490047 32743 leader_election.cc:288] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:41.490104 32751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:41.490267 32743 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:41.490294 32751 raft_consensus.cc:686] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [term 1 LEADER]: Becoming Leader. State: Replica: eee3abc0d808447b89cadeba3b3d3d1a, State: Running, Role: LEADER
I20260809 16:38:41.490423 32751 consensus_queue.cc:237] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } }
I20260809 16:38:41.490787 32753 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee3abc0d808447b89cadeba3b3d3d1a. Latest consensus state: current_term: 1 leader_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } } }
I20260809 16:38:41.490808 32752 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3abc0d808447b89cadeba3b3d3d1a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41653 } } }
I20260809 16:38:41.490931 32752 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:41.491158 32753 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:41.491346 32757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:41.491858 32757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:41.492676 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 32633
I20260809 16:38:41.492800 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:41.493026 31807 external_mini_cluster.cc:1428] Setting key 9cc4b30d369815c608bc6360355a66d0
I20260809 16:38:41.493510 32757 catalog_manager.cc:1347] Generated new cluster ID: 93a106760e234befb88e47d3b67ce200
I20260809 16:38:41.493561 32757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[32612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:38:41.503824 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:41.493455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43631 (local address 127.0.0.1:41653)
0809 16:38:41.495645 (+ 2190us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:41.495653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:41.495679 (+ 26us) server_negotiation.cc:407] Connection header received
0809 16:38:41.495767 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:41.495772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:41.495819 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:41.495925 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:41.496552 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:41.497470 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:41.498208 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:41.498617 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:41.498711 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:41.500883 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:41.500887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:41.500887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:41.500898 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:41.502397 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:41.502721 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:41.502724 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:41.502724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:41.502807 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:41.502977 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:41.502978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:41.502979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:41.503211 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:41.503386 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:41.503479 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 16:38:41.503616 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2152,"thread_start_us":81,"threads_started":1}
I20260809 16:38:41.504670 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:41.509148 32757 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:41.509788 32757 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:41.512841 312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:41.505890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59983 (local address 127.0.0.1:41653)
0809 16:38:41.505931 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:41.505933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:41.507435 (+ 1502us) server_negotiation.cc:407] Connection header received
0809 16:38:41.507490 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:41.507497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:41.507508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:41.507552 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:41.508013 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:41.508499 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:41.509301 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:41.509525 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:41.509578 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:41.510615 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:41.510616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:41.510616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:41.510623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:41.511870 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:41.512173 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:41.512174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:41.512175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:41.512227 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:41.512389 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:41.512390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:41.512390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:41.512490 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:41.512578 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:41.512677 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:38:41.512760 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:38:41.517611 32757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eee3abc0d808447b89cadeba3b3d3d1a: Generated new TSK 0
I20260809 16:38:41.517779 32757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:41.521546 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 32633
2026-08-09T16:38:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1416 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[324](info): set up 2 sockets
Aug 09 16:38:41 dist-test-slave-21b9 krb5kdc[324](info): commencing operation
krb5kdc: starting...
W20260809 16:38:42.705653 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:42 dist-test-slave-21b9 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:42.821544 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:42.902223 345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:42.902419 345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:42.902441 345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:42.903517 345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:42.903551 345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:42.903564 345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:42.903575 345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:42.903594 345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:42.904975 345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:42.905159 345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:42.905321 345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:42.909731 353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:42.909834 345 server_base.cc:1034] running on GCE node
W20260809 16:38:42.909765 355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:42.909729 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:42.910140 345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:42.910388 345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:42.911520 345 hybrid_clock.cc:648] HybridClock initialized: now 1786293522911501 us; error 34 us; skew 500 ppm
Aug 09 16:38:42 dist-test-slave-21b9 krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293522, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:42.916169 345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:42.916595 345 webserver.cc:460] Webserver started at http://127.0.0.1:36429/ using document root <none> and password file <none>
I20260809 16:38:42.916781 345 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:42.916828 345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:42.916951 345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:42.917891 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "96f574bb6489401cbc084384822601e6"
format_stamp: "Formatted at 2026-08-09 16:38:42 on dist-test-slave-21b9"
server_key: "6c4db6f32e15e2def4d8e55b0d7017f6"
server_key_iv: "04a32d9272416df17799125af42d568a"
server_key_version: "encryptionkey@0"
I20260809 16:38:42.918198 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "96f574bb6489401cbc084384822601e6"
format_stamp: "Formatted at 2026-08-09 16:38:42 on dist-test-slave-21b9"
server_key: "6c4db6f32e15e2def4d8e55b0d7017f6"
server_key_iv: "04a32d9272416df17799125af42d568a"
server_key_version: "encryptionkey@0"
I20260809 16:38:42.919428 345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:42.920063 372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:42.920203 345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:42.920295 345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "96f574bb6489401cbc084384822601e6"
format_stamp: "Formatted at 2026-08-09 16:38:42 on dist-test-slave-21b9"
server_key: "6c4db6f32e15e2def4d8e55b0d7017f6"
server_key_iv: "04a32d9272416df17799125af42d568a"
server_key_version: "encryptionkey@0"
I20260809 16:38:42.920360 345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:42.934732 345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:42.936291 345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:42.936411 345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:42.940137 345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41881
I20260809 16:38:42.940181 457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41881 every 8 connection(s)
I20260809 16:38:42.940478 345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:42.941228 458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:42.943336 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6: Bootstrap starting.
I20260809 16:38:42.943867 458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:42.944123 458 log.cc:824] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:42.944736 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6: No bootstrap required, opened a new log
I20260809 16:38:42.945959 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 345
I20260809 16:38:42.946079 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:42.946272 31807 external_mini_cluster.cc:1428] Setting key 46679cd9043fc8f4def2cf71275a3ddc
I20260809 16:38:42.946485 458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260809 16:38:42.946616 458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:42.946650 458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96f574bb6489401cbc084384822601e6, State: Initialized, Role: FOLLOWER
I20260809 16:38:42.946755 458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260809 16:38:42.946825 458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:42.946853 458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:42.946892 458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:42.947736 458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260809 16:38:42.947821 458 leader_election.cc:302] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96f574bb6489401cbc084384822601e6; no voters:
I20260809 16:38:42.947963 458 leader_election.cc:288] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:42.947996 465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:42.948120 465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [term 1 LEADER]: Becoming Leader. State: Replica: 96f574bb6489401cbc084384822601e6, State: Running, Role: LEADER
I20260809 16:38:42.948217 458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:42.948256 465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } }
I20260809 16:38:42.948632 465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96f574bb6489401cbc084384822601e6. Latest consensus state: current_term: 1 leader_uuid: "96f574bb6489401cbc084384822601e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } } }
I20260809 16:38:42.948736 465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:42.948690 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96f574bb6489401cbc084384822601e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96f574bb6489401cbc084384822601e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41881 } } }
I20260809 16:38:42.948808 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:42.949061 477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:42.949620 477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:42.951143 477 catalog_manager.cc:1347] Generated new cluster ID: 814fdc7566d34da9b74a86612094df97
I20260809 16:38:42.951196 477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:42.951737 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:42.946725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56421 (local address 127.0.0.1:41881)
0809 16:38:42.946904 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:42.946913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:42.946945 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:38:42.947062 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:42.947068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:42.947114 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:42.947223 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:42.947476 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:42.948616 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:42.949727 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:42.950317 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:42.950418 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:42.951064 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:42.951067 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:42.951068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:42.951246 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:42.951304 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:42.951320 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:42.951379 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:42.951484 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 16:38:42.951548 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260809 16:38:42.952489 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:42.957695 463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:42.953712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34037 (local address 127.0.0.1:41881)
0809 16:38:42.953742 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:42.953744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:42.955446 (+ 1702us) server_negotiation.cc:407] Connection header received
0809 16:38:42.955515 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:42.955516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:42.955526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:42.955570 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:42.955679 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:42.956102 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:42.956738 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:42.956915 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:42.956965 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:42.957489 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:42.957490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:42.957490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:42.957496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:42.957507 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:42.957511 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:42.957555 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:42.957606 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 16:38:42.957627 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:38:42.990505 477 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:42.991011 477 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:42.997155 477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96f574bb6489401cbc084384822601e6: Generated new TSK 0
I20260809 16:38:42.997301 477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:43.003850 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 345
2026-08-09T16:38:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:43 dist-test-slave-21b9 krb5kdc[506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:43 dist-test-slave-21b9 krb5kdc[506](info): set up 2 sockets
Aug 09 16:38:43 dist-test-slave-21b9 krb5kdc[506](info): commencing operation
krb5kdc: starting...
W20260809 16:38:44.168201 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:44 dist-test-slave-21b9 krb5kdc[506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:44.287516 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:44.369295 523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:44.369490 523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:44.369511 523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:44.370635 523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:44.370677 523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:44.370690 523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:44.370702 523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:44.370712 523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:44.372210 523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:44.372440 523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:44.372651 523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:44.377386 535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:44.377524 523 server_base.cc:1034] running on GCE node
W20260809 16:38:44.377395 538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:44.377367 533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:44.377833 523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:44.378098 523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:44.379237 523 hybrid_clock.cc:648] HybridClock initialized: now 1786293524379217 us; error 32 us; skew 500 ppm
Aug 09 16:38:44 dist-test-slave-21b9 krb5kdc[506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293524, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:44.381635 523 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:44.382061 523 webserver.cc:460] Webserver started at http://127.0.0.1:44059/ using document root <none> and password file <none>
I20260809 16:38:44.382282 523 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:44.382331 523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:44.382454 523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:44.383514 523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "5854e9fc679f48cf94cd908bd72e9d04"
format_stamp: "Formatted at 2026-08-09 16:38:44 on dist-test-slave-21b9"
server_key: "791ed1cf92ea990deaba82c24accc7d7"
server_key_iv: "4bc0e862a916f665fd2d1b779651199a"
server_key_version: "encryptionkey@0"
I20260809 16:38:44.383841 523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "5854e9fc679f48cf94cd908bd72e9d04"
format_stamp: "Formatted at 2026-08-09 16:38:44 on dist-test-slave-21b9"
server_key: "791ed1cf92ea990deaba82c24accc7d7"
server_key_iv: "4bc0e862a916f665fd2d1b779651199a"
server_key_version: "encryptionkey@0"
I20260809 16:38:44.385074 523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:44.385776 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:44.385924 523 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:44.386006 523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "5854e9fc679f48cf94cd908bd72e9d04"
format_stamp: "Formatted at 2026-08-09 16:38:44 on dist-test-slave-21b9"
server_key: "791ed1cf92ea990deaba82c24accc7d7"
server_key_iv: "4bc0e862a916f665fd2d1b779651199a"
server_key_version: "encryptionkey@0"
I20260809 16:38:44.386067 523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:44.389319 523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:44.390995 523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:44.391140 523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:44.394881 523 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44781
I20260809 16:38:44.394932 640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44781 every 8 connection(s)
I20260809 16:38:44.395229 523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:44.396133 642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:44.398308 642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: Bootstrap starting.
I20260809 16:38:44.398860 642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:44.399111 642 log.cc:824] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:44.399834 642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: No bootstrap required, opened a new log
I20260809 16:38:44.401396 642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } }
I20260809 16:38:44.401527 642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:44.401548 642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5854e9fc679f48cf94cd908bd72e9d04, State: Initialized, Role: FOLLOWER
I20260809 16:38:44.401691 642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } }
I20260809 16:38:44.401767 642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:44.401799 642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:44.401844 642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:44.402297 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 523
I20260809 16:38:44.402420 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:44.402434 642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } }
I20260809 16:38:44.402539 642 leader_election.cc:302] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5854e9fc679f48cf94cd908bd72e9d04; no voters:
I20260809 16:38:44.402637 31807 external_mini_cluster.cc:1428] Setting key 5334fbe5b8c0b327c090a8e860e6edfd
I20260809 16:38:44.402700 642 leader_election.cc:288] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:44.402748 649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:44.402930 649 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [term 1 LEADER]: Becoming Leader. State: Replica: 5854e9fc679f48cf94cd908bd72e9d04, State: Running, Role: LEADER
I20260809 16:38:44.402951 642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:44.403115 649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } }
I20260809 16:38:44.403745 650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5854e9fc679f48cf94cd908bd72e9d04. Latest consensus state: current_term: 1 leader_uuid: "5854e9fc679f48cf94cd908bd72e9d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } } }
I20260809 16:38:44.403761 651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5854e9fc679f48cf94cd908bd72e9d04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5854e9fc679f48cf94cd908bd72e9d04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44781 } } }
I20260809 16:38:44.403836 650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:44.403864 651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04 [sys.catalog]: This master's current role is: LEADER
W20260809 16:38:44.404793 679 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:38:44.405069 679 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:38:44.405184 667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:44.405375 667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:44.405525 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:44.403393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44961 (local address 127.0.0.1:44781)
0809 16:38:44.404058 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:44.404067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:44.404101 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:38:44.404243 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:44.404250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:44.404283 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:44.404703 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:44.405096 (+ 393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:44.405100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:44.405100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:44.405104 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:44.405141 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:44.405155 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:44.405217 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:44.405297 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 16:38:44.405339 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":88,"threads_started":1}
I20260809 16:38:44.406425 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:44.411676 667 catalog_manager.cc:1347] Generated new cluster ID: a416387368df402cb04dc04c2aff68a2
I20260809 16:38:44.411736 667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:44.412720 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:44.407912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35555 (local address 127.0.0.1:44781)
0809 16:38:44.407963 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:44.407965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:44.411757 (+ 3792us) server_negotiation.cc:407] Connection header received
0809 16:38:44.411868 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:44.411869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:44.411874 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:44.411942 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:44.412517 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:44.412518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:44.412518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:44.412522 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:44.412538 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:44.412542 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:44.412596 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:44.412646 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 16:38:44.412659 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 16:38:44.425803 667 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:44.425971 667 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:44.433495 667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5854e9fc679f48cf94cd908bd72e9d04: Generated new TSK 0
I20260809 16:38:44.433672 667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:44.437747 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 523
2026-08-09T16:38:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:44 dist-test-slave-21b9 krb5kdc[696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:44 dist-test-slave-21b9 krb5kdc[696](info): set up 2 sockets
Aug 09 16:38:44 dist-test-slave-21b9 krb5kdc[696](info): commencing operation
krb5kdc: starting...
W20260809 16:38:45.601613 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:45.715040 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:45.795519 715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:45.795724 715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:45.795756 715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:45.796871 715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:45.796921 715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:45.796938 715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:45.796953 715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:45.796967 715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:45.798408 715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:45.798655 715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:45.798862 715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:45.803506 722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:45.803536 721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:45.803695 715 server_base.cc:1034] running on GCE node
W20260809 16:38:45.803562 725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:45.803982 715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:45.804271 715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:45.805445 715 hybrid_clock.cc:648] HybridClock initialized: now 1786293525805424 us; error 35 us; skew 500 ppm
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293525, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:45.807896 715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:45.808270 715 webserver.cc:460] Webserver started at http://127.0.0.1:43253/ using document root <none> and password file <none>
I20260809 16:38:45.808441 715 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:45.808488 715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:45.808581 715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:45.809567 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "cb6ef78477ff4ea09b37185e6b9eaba8"
format_stamp: "Formatted at 2026-08-09 16:38:45 on dist-test-slave-21b9"
server_key: "5eed7873eb4da178d35da4df557148f8"
server_key_iv: "cc120d861e0db87776031115d1685461"
server_key_version: "encryptionkey@0"
I20260809 16:38:45.809856 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "cb6ef78477ff4ea09b37185e6b9eaba8"
format_stamp: "Formatted at 2026-08-09 16:38:45 on dist-test-slave-21b9"
server_key: "5eed7873eb4da178d35da4df557148f8"
server_key_iv: "cc120d861e0db87776031115d1685461"
server_key_version: "encryptionkey@0"
I20260809 16:38:45.811040 715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:45.811765 741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:45.811915 715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:45.812002 715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "cb6ef78477ff4ea09b37185e6b9eaba8"
format_stamp: "Formatted at 2026-08-09 16:38:45 on dist-test-slave-21b9"
server_key: "5eed7873eb4da178d35da4df557148f8"
server_key_iv: "cc120d861e0db87776031115d1685461"
server_key_version: "encryptionkey@0"
I20260809 16:38:45.812065 715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:45.825377 715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:45.827451 715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:45.827590 715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:45.831338 715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43525
I20260809 16:38:45.831369 829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43525 every 8 connection(s)
I20260809 16:38:45.831699 715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:45.832510 831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:45.834697 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: Bootstrap starting.
I20260809 16:38:45.835222 831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:45.835502 831 log.cc:824] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:45.836128 831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: No bootstrap required, opened a new log
I20260809 16:38:45.837651 831 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } }
I20260809 16:38:45.837782 831 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:45.837803 831 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6ef78477ff4ea09b37185e6b9eaba8, State: Initialized, Role: FOLLOWER
I20260809 16:38:45.837913 831 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } }
I20260809 16:38:45.837988 831 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:45.838016 831 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:45.838053 831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:45.838510 831 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } }
I20260809 16:38:45.838596 831 leader_election.cc:302] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb6ef78477ff4ea09b37185e6b9eaba8; no voters:
I20260809 16:38:45.838721 831 leader_election.cc:288] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:45.838763 839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:45.838930 831 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:45.838948 839 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [term 1 LEADER]: Becoming Leader. State: Replica: cb6ef78477ff4ea09b37185e6b9eaba8, State: Running, Role: LEADER
I20260809 16:38:45.839088 839 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } }
I20260809 16:38:45.839627 846 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb6ef78477ff4ea09b37185e6b9eaba8. Latest consensus state: current_term: 1 leader_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } } }
I20260809 16:38:45.839722 846 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:45.839725 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 715
I20260809 16:38:45.839836 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:45.840010 31807 external_mini_cluster.cc:1428] Setting key 74c75259c1678b52f9778ef57f5b62d2
I20260809 16:38:45.840755 841 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb6ef78477ff4ea09b37185e6b9eaba8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43525 } } }
I20260809 16:38:45.840879 841 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8 [sys.catalog]: This master's current role is: LEADER
W20260809 16:38:45.841552 870 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:38:45.841607 870 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:38:45.841660 852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:45.841822 852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:45.843204 852 catalog_manager.cc:1347] Generated new cluster ID: 32d0bfea07574d4c965f337493d154ff
I20260809 16:38:45.843255 852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[696](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:38:45.849670 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:45.840644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:56603 (local address 127.0.0.1:43525)
0809 16:38:45.840797 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:45.840806 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:45.840837 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:38:45.840953 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:45.840960 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:45.841040 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:45.841957 (+ 917us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:45.842052 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:45.843212 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:45.843969 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:45.844389 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:45.844482 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:45.846652 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:45.846656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:45.846657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:45.846669 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:45.848188 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:45.848549 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:45.848552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:45.848553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:45.848645 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:45.848843 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:45.848844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:45.848844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:45.849083 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:45.849251 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:45.849361 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 16:38:45.849461 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":85,"threads_started":1}
I20260809 16:38:45.850850 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:45.855564 852 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:45.857306 852 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:45.857740 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:45.852087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:54135 (local address 127.0.0.1:43525)
0809 16:38:45.852121 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:45.852123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:45.852235 (+ 112us) server_negotiation.cc:407] Connection header received
0809 16:38:45.852286 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:45.852287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:45.852296 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:45.852338 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:45.852795 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:45.853251 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:45.853884 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:45.854099 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:45.854151 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:45.855333 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:45.855334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:45.855335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:45.855341 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:45.856703 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:45.857047 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:45.857048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:45.857049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:45.857100 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:45.857280 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:45.857281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:45.857281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:45.857393 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:45.857505 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:45.857604 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:38:45.857662 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:38:45.877774 852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb6ef78477ff4ea09b37185e6b9eaba8: Generated new TSK 0
I20260809 16:38:45.877982 852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:45.886164 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 715
W20260809 16:38:45.901480 693 connection.cc:535] client connection to 127.0.0.1:43525 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43525): Connection reset by peer (error 104)
2026-08-09T16:38:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[884](info): set up 2 sockets
Aug 09 16:38:45 dist-test-slave-21b9 krb5kdc[884](info): commencing operation
krb5kdc: starting...
W20260809 16:38:47.039075 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:47 dist-test-slave-21b9 krb5kdc[884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:47.151783 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:47.231761 907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:47.231950 907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:47.231972 907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:47.233026 907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:47.233062 907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:47.233073 907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:47.233084 907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:47.233109 907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:47.234503 907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:47.234694 907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:47.234874 907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:47.239455 919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:47.239462 914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:47.239462 915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:47.239640 907 server_base.cc:1034] running on GCE node
I20260809 16:38:47.239876 907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:47.240146 907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:47.241291 907 hybrid_clock.cc:648] HybridClock initialized: now 1786293527241275 us; error 29 us; skew 500 ppm
Aug 09 16:38:47 dist-test-slave-21b9 krb5kdc[884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293527, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:47.243752 907 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:47.244158 907 webserver.cc:460] Webserver started at http://127.0.0.1:34609/ using document root <none> and password file <none>
I20260809 16:38:47.244376 907 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:47.244426 907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:47.244549 907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:47.245560 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "197124216e884d149daac5c8c65192e7"
format_stamp: "Formatted at 2026-08-09 16:38:47 on dist-test-slave-21b9"
server_key: "94b4a57bcb925f7ed75f6aea0d12e5eb"
server_key_iv: "6272bd4fbb63b97744f7c6a097738c49"
server_key_version: "encryptionkey@0"
I20260809 16:38:47.245889 907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "197124216e884d149daac5c8c65192e7"
format_stamp: "Formatted at 2026-08-09 16:38:47 on dist-test-slave-21b9"
server_key: "94b4a57bcb925f7ed75f6aea0d12e5eb"
server_key_iv: "6272bd4fbb63b97744f7c6a097738c49"
server_key_version: "encryptionkey@0"
I20260809 16:38:47.247184 907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:47.248003 937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:47.248170 907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:47.248265 907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "197124216e884d149daac5c8c65192e7"
format_stamp: "Formatted at 2026-08-09 16:38:47 on dist-test-slave-21b9"
server_key: "94b4a57bcb925f7ed75f6aea0d12e5eb"
server_key_iv: "6272bd4fbb63b97744f7c6a097738c49"
server_key_version: "encryptionkey@0"
I20260809 16:38:47.248338 907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:47.284317 907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:47.286371 907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:47.286525 907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:47.290815 907 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33119
I20260809 16:38:47.290854 1028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33119 every 8 connection(s)
I20260809 16:38:47.291204 907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:47.292145 1030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:47.294441 1030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7: Bootstrap starting.
I20260809 16:38:47.295024 1030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:47.295315 1030 log.cc:824] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:47.295994 1030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7: No bootstrap required, opened a new log
I20260809 16:38:47.296494 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 907
I20260809 16:38:47.296640 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:47.296831 31807 external_mini_cluster.cc:1428] Setting key be9e8f51e1b87554fd7540c02738cfc1
I20260809 16:38:47.298110 1030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } }
I20260809 16:38:47.298244 1030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:47.298265 1030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 197124216e884d149daac5c8c65192e7, State: Initialized, Role: FOLLOWER
I20260809 16:38:47.298370 1030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } }
I20260809 16:38:47.298439 1030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:47.298461 1030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:47.298506 1030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:47.299106 1030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } }
I20260809 16:38:47.299196 1030 leader_election.cc:302] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 197124216e884d149daac5c8c65192e7; no voters:
I20260809 16:38:47.299367 1030 leader_election.cc:288] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:47.299412 1038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:47.299539 1038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [term 1 LEADER]: Becoming Leader. State: Replica: 197124216e884d149daac5c8c65192e7, State: Running, Role: LEADER
I20260809 16:38:47.299628 1030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:47.299621 1038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } }
I20260809 16:38:47.300050 1041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "197124216e884d149daac5c8c65192e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } } }
I20260809 16:38:47.300056 1040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 197124216e884d149daac5c8c65192e7. Latest consensus state: current_term: 1 leader_uuid: "197124216e884d149daac5c8c65192e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "197124216e884d149daac5c8c65192e7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33119 } } }
I20260809 16:38:47.300143 1041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:47.300145 1040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:47.300390 1049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:47.300889 1049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:47.302711 1049 catalog_manager.cc:1347] Generated new cluster ID: aec1c5dba5d744548b61f1974ef3902e
I20260809 16:38:47.302773 1049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:47.303639 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:47.297342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:39255 (local address 127.0.0.1:33119)
0809 16:38:47.297522 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:47.297530 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:47.297557 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:38:47.298296 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:47.298309 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:47.298381 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:47.298538 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:47.298787 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:47.299968 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:47.301286 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:47.301818 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:47.301958 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:47.302809 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:47.302813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:47.302813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:47.303034 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:47.303072 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:47.303087 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:47.303338 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:47.303406 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 16:38:47.303434 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260809 16:38:47.304569 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:47.309569 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:47.306005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:58591 (local address 127.0.0.1:33119)
0809 16:38:47.306045 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:47.306047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:47.306162 (+ 115us) server_negotiation.cc:407] Connection header received
0809 16:38:47.306250 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:47.306251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:47.306260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:47.306288 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:47.306481 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:47.307156 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:47.308181 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:47.308481 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:47.308520 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:47.309285 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:47.309286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:47.309287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:47.309291 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:47.309306 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:47.309310 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:47.309345 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:47.309487 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:38:47.309497 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:38:47.322350 1049 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:47.323021 1049 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:47.326952 1049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 197124216e884d149daac5c8c65192e7: Generated new TSK 0
I20260809 16:38:47.327136 1049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:47.330871 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 907
W20260809 16:38:47.344686 882 connection.cc:535] client connection to 127.0.0.1:33119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33119): Connection reset by peer (error 104)
2026-08-09T16:38:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:47 dist-test-slave-21b9 krb5kdc[1079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:47 dist-test-slave-21b9 krb5kdc[1079](info): set up 2 sockets
Aug 09 16:38:47 dist-test-slave-21b9 krb5kdc[1079](info): commencing operation
krb5kdc: starting...
W20260809 16:38:48.493862 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:48 dist-test-slave-21b9 krb5kdc[1079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:48.607761 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:48.687847 1110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:48.688050 1110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:48.688072 1110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:48.689194 1110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:48.689239 1110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:48.689251 1110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:48.689262 1110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:48.689273 1110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:48.690711 1110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:48.690927 1110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:48.691138 1110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:48.695772 1124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:48.695829 1120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:48.695932 1110 server_base.cc:1034] running on GCE node
W20260809 16:38:48.695848 1118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:48.696228 1110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:48.696484 1110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:48.697659 1110 hybrid_clock.cc:648] HybridClock initialized: now 1786293528697648 us; error 30 us; skew 500 ppm
Aug 09 16:38:48 dist-test-slave-21b9 krb5kdc[1079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:48.703352 1110 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:48.703822 1110 webserver.cc:460] Webserver started at http://127.0.0.1:40813/ using document root <none> and password file <none>
I20260809 16:38:48.704027 1110 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:48.704070 1110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:48.704195 1110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:48.705227 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "9ff8eb64d8144fd58da35976e46f3f00"
format_stamp: "Formatted at 2026-08-09 16:38:48 on dist-test-slave-21b9"
server_key: "8b2317b209cf396ab4e52ac824f21553"
server_key_iv: "785afe8774be47d5432a083541ec770b"
server_key_version: "encryptionkey@0"
I20260809 16:38:48.705557 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "9ff8eb64d8144fd58da35976e46f3f00"
format_stamp: "Formatted at 2026-08-09 16:38:48 on dist-test-slave-21b9"
server_key: "8b2317b209cf396ab4e52ac824f21553"
server_key_iv: "785afe8774be47d5432a083541ec770b"
server_key_version: "encryptionkey@0"
I20260809 16:38:48.706779 1110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:48.707454 1141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:48.707593 1110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:48.707680 1110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "9ff8eb64d8144fd58da35976e46f3f00"
format_stamp: "Formatted at 2026-08-09 16:38:48 on dist-test-slave-21b9"
server_key: "8b2317b209cf396ab4e52ac824f21553"
server_key_iv: "785afe8774be47d5432a083541ec770b"
server_key_version: "encryptionkey@0"
I20260809 16:38:48.707739 1110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:48.710862 1110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:48.712743 1110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:48.712873 1110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:48.716744 1110 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39745
I20260809 16:38:48.716775 1232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39745 every 8 connection(s)
I20260809 16:38:48.717078 1110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:48.717893 1234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:48.720129 1234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00: Bootstrap starting.
I20260809 16:38:48.720657 1234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:48.720901 1234 log.cc:824] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:48.721513 1234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00: No bootstrap required, opened a new log
I20260809 16:38:48.722330 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1110
I20260809 16:38:48.722431 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:48.722600 31807 external_mini_cluster.cc:1428] Setting key a1093d9823e513409ecf00e20ed83f79
I20260809 16:38:48.723196 1234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } }
I20260809 16:38:48.723364 1234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:48.723413 1234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ff8eb64d8144fd58da35976e46f3f00, State: Initialized, Role: FOLLOWER
I20260809 16:38:48.723542 1234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } }
I20260809 16:38:48.723623 1234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:48.723662 1234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:48.723708 1234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:48.724730 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:48.723102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:37191 (local address 127.0.0.1:39745)
0809 16:38:48.723289 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:48.723299 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:48.723376 (+ 77us) server_negotiation.cc:407] Connection header received
0809 16:38:48.723562 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:48.723570 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:48.723599 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:48.723742 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:48.724340 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:48.724344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:48.724344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:48.724348 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:48.724388 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:48.724401 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:48.724474 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:48.724512 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:38:48.724554 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260809 16:38:48.724716 1234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } }
I20260809 16:38:48.724831 1234 leader_election.cc:302] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ff8eb64d8144fd58da35976e46f3f00; no voters:
I20260809 16:38:48.724977 1234 leader_election.cc:288] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:48.725023 1245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:48.725136 1245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [term 1 LEADER]: Becoming Leader. State: Replica: 9ff8eb64d8144fd58da35976e46f3f00, State: Running, Role: LEADER
I20260809 16:38:48.725221 1234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:48.725212 1245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } }
I20260809 16:38:48.725521 1246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ff8eb64d8144fd58da35976e46f3f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } } }
I20260809 16:38:48.725575 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:48.725608 1246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:48.725829 1248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ff8eb64d8144fd58da35976e46f3f00. Latest consensus state: current_term: 1 leader_uuid: "9ff8eb64d8144fd58da35976e46f3f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ff8eb64d8144fd58da35976e46f3f00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39745 } } }
I20260809 16:38:48.726168 1252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:48.726814 1252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:48.727509 1248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:48.728402 1243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:48.727089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:48783 (local address 127.0.0.1:39745)
0809 16:38:48.727121 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:48.727122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:48.727355 (+ 233us) server_negotiation.cc:407] Connection header received
0809 16:38:48.727516 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:48.727518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:48.727522 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:48.727580 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:48.728188 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:48.728189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:48.728190 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:48.728194 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:48.728208 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:48.728213 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:48.728239 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:48.728334 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 16:38:48.728347 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:38:48.728449 1252 catalog_manager.cc:1347] Generated new cluster ID: 2de4933a47714b6caa420c452294d1c1
I20260809 16:38:48.728483 1252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:48.740743 1252 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:48.740917 1252 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:48.744118 1252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ff8eb64d8144fd58da35976e46f3f00: Generated new TSK 0
I20260809 16:38:48.744288 1252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:48.750782 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1110
2026-08-09T16:38:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:48 dist-test-slave-21b9 krb5kdc[1280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:48 dist-test-slave-21b9 krb5kdc[1280](info): set up 2 sockets
Aug 09 16:38:48 dist-test-slave-21b9 krb5kdc[1280](info): commencing operation
krb5kdc: starting...
W20260809 16:38:49.912758 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:49 dist-test-slave-21b9 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:50.031934 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:50.112257 1309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:50.112462 1309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:50.112493 1309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:50.113610 1309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:50.113651 1309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:50.113670 1309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:50.113686 1309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:50.113700 1309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:50.115150 1309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:50.115417 1309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:50.115619 1309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:50.120160 1320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:50.120170 1325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:50.120204 1319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:50.120365 1309 server_base.cc:1034] running on GCE node
I20260809 16:38:50.120587 1309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:50.120819 1309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:50.121949 1309 hybrid_clock.cc:648] HybridClock initialized: now 1786293530121936 us; error 30 us; skew 500 ppm
Aug 09 16:38:50 dist-test-slave-21b9 krb5kdc[1280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:50.124344 1309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:50.124753 1309 webserver.cc:460] Webserver started at http://127.0.0.1:37147/ using document root <none> and password file <none>
I20260809 16:38:50.124944 1309 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:50.125000 1309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:50.125124 1309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:50.126116 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "f08c11ee39b845db9be6cc4c337a6e1f"
format_stamp: "Formatted at 2026-08-09 16:38:50 on dist-test-slave-21b9"
server_key: "013ad3d59ce7bcc1b930a483c51a277d"
server_key_iv: "4382e9e603d0df7e245b3b1961f41395"
server_key_version: "encryptionkey@0"
I20260809 16:38:50.126431 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "f08c11ee39b845db9be6cc4c337a6e1f"
format_stamp: "Formatted at 2026-08-09 16:38:50 on dist-test-slave-21b9"
server_key: "013ad3d59ce7bcc1b930a483c51a277d"
server_key_iv: "4382e9e603d0df7e245b3b1961f41395"
server_key_version: "encryptionkey@0"
I20260809 16:38:50.127701 1309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:38:50.128329 1337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:50.128480 1309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:38:50.128562 1309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "f08c11ee39b845db9be6cc4c337a6e1f"
format_stamp: "Formatted at 2026-08-09 16:38:50 on dist-test-slave-21b9"
server_key: "013ad3d59ce7bcc1b930a483c51a277d"
server_key_iv: "4382e9e603d0df7e245b3b1961f41395"
server_key_version: "encryptionkey@0"
I20260809 16:38:50.128618 1309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:50.143805 1309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:50.145510 1309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:50.145639 1309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:50.149464 1309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41233
I20260809 16:38:50.149497 1423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41233 every 8 connection(s)
I20260809 16:38:50.149807 1309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:50.150638 1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:50.152941 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f: Bootstrap starting.
I20260809 16:38:50.153499 1424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:50.153777 1424 log.cc:824] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:50.154407 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f: No bootstrap required, opened a new log
I20260809 16:38:50.156034 1424 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } }
I20260809 16:38:50.156169 1424 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:50.156191 1424 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08c11ee39b845db9be6cc4c337a6e1f, State: Initialized, Role: FOLLOWER
I20260809 16:38:50.156334 1424 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } }
I20260809 16:38:50.156399 1424 raft_consensus.cc:388] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:50.156427 1424 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:50.156472 1424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:50.156891 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1309
I20260809 16:38:50.157017 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:50.157075 1424 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } }
I20260809 16:38:50.157183 1424 leader_election.cc:302] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f08c11ee39b845db9be6cc4c337a6e1f; no voters:
I20260809 16:38:50.157208 31807 external_mini_cluster.cc:1428] Setting key 2b10f9ffb6cd96eb931a8ea9ef300d57
I20260809 16:38:50.157341 1424 leader_election.cc:288] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:50.157379 1427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:50.157542 1427 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [term 1 LEADER]: Becoming Leader. State: Replica: f08c11ee39b845db9be6cc4c337a6e1f, State: Running, Role: LEADER
I20260809 16:38:50.157594 1424 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:50.157652 1427 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } }
I20260809 16:38:50.157979 1427 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08c11ee39b845db9be6cc4c337a6e1f. Latest consensus state: current_term: 1 leader_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } } }
I20260809 16:38:50.158056 1427 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:50.158209 1428 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f08c11ee39b845db9be6cc4c337a6e1f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41233 } } }
I20260809 16:38:50.158245 1440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:50.158304 1428 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:50.159523 1440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:50.160717 1454 mvcc.cc:205] Tried to move back new op lower bound from 7316658299536789504 to 7316658299526156288. Current Snapshot: MvccSnapshot[applied={T|T < 7316658299536789504}]
I20260809 16:38:50.161056 1440 catalog_manager.cc:1347] Generated new cluster ID: 4a46b410810348b39bc90e28674e2564
I20260809 16:38:50.161103 1440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:50.174922 1440 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:50.175566 1440 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 16:38:50 dist-test-slave-21b9 krb5kdc[1280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 16:38:50.179978 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:50.157849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:36305 (local address 127.0.0.1:41233)
0809 16:38:50.158408 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:50.158419 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:50.158452 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:38:50.158571 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:50.158579 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:50.158641 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:50.158819 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:50.159569 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:50.172020 (+ 12451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:50.173072 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:50.173692 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:50.173804 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:50.176623 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:50.176627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:50.176627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:50.176639 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:50.178414 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:50.178832 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:50.178834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:50.178835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:50.178926 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:50.179134 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:50.179134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:50.179134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:50.179404 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:50.179566 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:50.179665 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 16:38:50.179758 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":77,"threads_started":1}
I20260809 16:38:50.180841 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:50.182622 1440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08c11ee39b845db9be6cc4c337a6e1f: Generated new TSK 0
I20260809 16:38:50.182776 1440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:50.187798 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:50.182292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:35175 (local address 127.0.0.1:41233)
0809 16:38:50.182329 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:50.182330 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:50.182414 (+ 84us) server_negotiation.cc:407] Connection header received
0809 16:38:50.182463 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:50.182464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:50.182475 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:50.182536 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:50.183098 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:50.183647 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:50.184335 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:50.184560 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:50.184615 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:50.185641 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:50.185642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:50.185642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:50.185648 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:50.186771 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:50.187095 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:50.187096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:50.187096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:50.187147 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:50.187351 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:50.187352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:50.187352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:50.187453 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:50.187559 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:50.187665 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 16:38:50.187721 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:38:50.189464 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1309
W20260809 16:38:50.208714 1278 connection.cc:535] client connection to 127.0.0.1:41233 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41233): Connection reset by peer (error 104)
2026-08-09T16:38:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:50 dist-test-slave-21b9 krb5kdc[1470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:50 dist-test-slave-21b9 krb5kdc[1470](info): set up 2 sockets
Aug 09 16:38:50 dist-test-slave-21b9 krb5kdc[1470](info): commencing operation
krb5kdc: starting...
W20260809 16:38:51.353928 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:51 dist-test-slave-21b9 krb5kdc[1470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:51.470512 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:51.550611 1499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:51.550814 1499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:51.550846 1499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:51.552043 1499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:51.552086 1499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:51.552106 1499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:51.552121 1499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:51.552137 1499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:51.553607 1499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45909
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:51.553871 1499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:51.554083 1499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:51.558595 1511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:51.558538 1515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:51.558539 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:51.558910 1499 server_base.cc:1034] running on GCE node
I20260809 16:38:51.559144 1499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:51.559437 1499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:51.560599 1499 hybrid_clock.cc:648] HybridClock initialized: now 1786293531560582 us; error 32 us; skew 500 ppm
Aug 09 16:38:51 dist-test-slave-21b9 krb5kdc[1470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:51.563103 1499 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:51.563557 1499 webserver.cc:460] Webserver started at http://127.0.0.1:39031/ using document root <none> and password file <none>
I20260809 16:38:51.563782 1499 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:51.563830 1499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:51.563984 1499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:51.565065 1499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "c70634b427f049b68d072007f332a445"
format_stamp: "Formatted at 2026-08-09 16:38:51 on dist-test-slave-21b9"
server_key: "bfa3f243dcd8d4ca059a8e1e584af561"
server_key_iv: "b94d7b70db851049ed9cbfdcc35077cf"
server_key_version: "encryptionkey@0"
I20260809 16:38:51.565393 1499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "c70634b427f049b68d072007f332a445"
format_stamp: "Formatted at 2026-08-09 16:38:51 on dist-test-slave-21b9"
server_key: "bfa3f243dcd8d4ca059a8e1e584af561"
server_key_iv: "b94d7b70db851049ed9cbfdcc35077cf"
server_key_version: "encryptionkey@0"
I20260809 16:38:51.566618 1499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:38:51.567262 1525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:51.567425 1499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:51.567504 1499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "c70634b427f049b68d072007f332a445"
format_stamp: "Formatted at 2026-08-09 16:38:51 on dist-test-slave-21b9"
server_key: "bfa3f243dcd8d4ca059a8e1e584af561"
server_key_iv: "b94d7b70db851049ed9cbfdcc35077cf"
server_key_version: "encryptionkey@0"
I20260809 16:38:51.567572 1499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:51.580456 1499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:51.582106 1499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:51.582242 1499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:51.586334 1499 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45909
I20260809 16:38:51.586380 1604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45909 every 8 connection(s)
I20260809 16:38:51.586720 1499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:51.587611 1606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:51.589854 1606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445: Bootstrap starting.
I20260809 16:38:51.590423 1606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:51.590687 1606 log.cc:824] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:51.591364 1606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445: No bootstrap required, opened a new log
I20260809 16:38:51.592926 1606 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } }
I20260809 16:38:51.593073 1606 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:51.593096 1606 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70634b427f049b68d072007f332a445, State: Initialized, Role: FOLLOWER
I20260809 16:38:51.593216 1606 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } }
I20260809 16:38:51.593282 1606 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:51.593304 1606 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:51.593350 1606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:51.593932 1606 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } }
I20260809 16:38:51.594048 1606 leader_election.cc:302] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70634b427f049b68d072007f332a445; no voters:
I20260809 16:38:51.594182 1606 leader_election.cc:288] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:51.594231 1611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:51.594396 1611 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [term 1 LEADER]: Becoming Leader. State: Replica: c70634b427f049b68d072007f332a445, State: Running, Role: LEADER
I20260809 16:38:51.594410 1606 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:51.594542 1611 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } }
I20260809 16:38:51.594964 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1499
I20260809 16:38:51.595090 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:51.595635 31807 external_mini_cluster.cc:1428] Setting key 9589d869f6f2fee02fb0a4347260df4b
I20260809 16:38:51.595587 1613 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70634b427f049b68d072007f332a445. Latest consensus state: current_term: 1 leader_uuid: "c70634b427f049b68d072007f332a445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } } }
I20260809 16:38:51.595521 1612 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70634b427f049b68d072007f332a445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70634b427f049b68d072007f332a445" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45909 } } }
I20260809 16:38:51.595727 1613 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:51.595736 1612 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:51.595948 1640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:51.596536 1640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:51.599704 1640 catalog_manager.cc:1347] Generated new cluster ID: 4481027ea3264fbc8089385cb58cc227
I20260809 16:38:51.599752 1640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:51.601984 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:51.596265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:34653 (local address 127.0.0.1:45909)
0809 16:38:51.597378 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:51.597387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:51.597417 (+ 30us) server_negotiation.cc:407] Connection header received
0809 16:38:51.597545 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:51.597552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:51.597608 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:51.597740 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:51.597930 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:51.599328 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:51.600181 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:51.600616 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:51.600713 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:51.601367 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:51.601372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:51.601372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:51.601539 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:51.601571 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:51.601582 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:51.601652 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:51.601755 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 16:38:51.601783 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1060,"thread_start_us":97,"threads_started":1}
I20260809 16:38:51.602984 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:51.610379 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:51.607722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:51881 (local address 127.0.0.1:45909)
0809 16:38:51.607757 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:51.607759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:51.607923 (+ 164us) server_negotiation.cc:407] Connection header received
0809 16:38:51.607971 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:51.607972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:51.607983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:51.608037 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:51.608163 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:51.608616 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:51.609301 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:51.609517 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:51.609572 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:51.610151 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:51.610153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:51.610153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:51.610157 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:51.610170 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:51.610174 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:51.610222 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:51.610287 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 16:38:51.610299 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:38:51.614315 1640 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:51.614789 1640 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:51.620152 1640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70634b427f049b68d072007f332a445: Generated new TSK 0
I20260809 16:38:51.620303 1640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:51.629017 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1499
W20260809 16:38:51.648092 1467 connection.cc:535] client connection to 127.0.0.1:45909 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45909): Connection reset by peer (error 104)
2026-08-09T16:38:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:51 dist-test-slave-21b9 krb5kdc[1667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:51 dist-test-slave-21b9 krb5kdc[1667](info): set up 2 sockets
Aug 09 16:38:51 dist-test-slave-21b9 krb5kdc[1667](info): commencing operation
krb5kdc: starting...
W20260809 16:38:52.796248 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:52 dist-test-slave-21b9 krb5kdc[1667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 16:38:52.909845 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:52.989228 1681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:52.989415 1681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:52.989447 1681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:52.990547 1681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:52.990586 1681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:52.990604 1681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:52.990620 1681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:52.990635 1681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:52.992070 1681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44311
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:52.992306 1681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:52.992503 1681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:52.997093 1692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:52.997098 1688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:52.997135 1690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:52.997329 1681 server_base.cc:1034] running on GCE node
I20260809 16:38:52.997594 1681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:52.997861 1681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:52.999006 1681 hybrid_clock.cc:648] HybridClock initialized: now 1786293532998995 us; error 30 us; skew 500 ppm
Aug 09 16:38:52 dist-test-slave-21b9 krb5kdc[1667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:53.001374 1681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 16:38:53.001818 1681 webserver.cc:460] Webserver started at http://127.0.0.1:44123/ using document root <none> and password file <none>
I20260809 16:38:53.002019 1681 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:53.002066 1681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:53.002194 1681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:53.003176 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "1ebecf8cbc984090b9a5207e29b993e6"
format_stamp: "Formatted at 2026-08-09 16:38:52 on dist-test-slave-21b9"
server_key: "f073ae690f2f5e9c01a53034fdce8ec7"
server_key_iv: "2248961c197d323438ad83fbc4f54645"
server_key_version: "encryptionkey@0"
I20260809 16:38:53.003556 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "1ebecf8cbc984090b9a5207e29b993e6"
format_stamp: "Formatted at 2026-08-09 16:38:52 on dist-test-slave-21b9"
server_key: "f073ae690f2f5e9c01a53034fdce8ec7"
server_key_iv: "2248961c197d323438ad83fbc4f54645"
server_key_version: "encryptionkey@0"
I20260809 16:38:53.004763 1681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:53.005412 1707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:53.005573 1681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:53.005658 1681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "1ebecf8cbc984090b9a5207e29b993e6"
format_stamp: "Formatted at 2026-08-09 16:38:52 on dist-test-slave-21b9"
server_key: "f073ae690f2f5e9c01a53034fdce8ec7"
server_key_iv: "2248961c197d323438ad83fbc4f54645"
server_key_version: "encryptionkey@0"
I20260809 16:38:53.005725 1681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:53.009028 1681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:53.010723 1681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:53.010848 1681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:53.014629 1681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44311
I20260809 16:38:53.014659 1807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44311 every 8 connection(s)
I20260809 16:38:53.014946 1681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:53.015789 1808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:53.017967 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6: Bootstrap starting.
I20260809 16:38:53.018502 1808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:53.018751 1808 log.cc:824] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:53.019405 1808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6: No bootstrap required, opened a new log
I20260809 16:38:53.020956 1808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } }
I20260809 16:38:53.021102 1808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:53.021126 1808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ebecf8cbc984090b9a5207e29b993e6, State: Initialized, Role: FOLLOWER
I20260809 16:38:53.021231 1808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } }
I20260809 16:38:53.021278 1808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:53.021296 1808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:53.021327 1808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:53.021799 1808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } }
I20260809 16:38:53.021883 1808 leader_election.cc:302] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ebecf8cbc984090b9a5207e29b993e6; no voters:
I20260809 16:38:53.022003 1808 leader_election.cc:288] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:53.022061 1811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:53.022233 1808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:53.022233 1811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [term 1 LEADER]: Becoming Leader. State: Replica: 1ebecf8cbc984090b9a5207e29b993e6, State: Running, Role: LEADER
I20260809 16:38:53.022403 1811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } }
I20260809 16:38:53.022868 1813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ebecf8cbc984090b9a5207e29b993e6. Latest consensus state: current_term: 1 leader_uuid: "1ebecf8cbc984090b9a5207e29b993e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } } }
I20260809 16:38:53.022949 1813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:53.023036 1812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ebecf8cbc984090b9a5207e29b993e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ebecf8cbc984090b9a5207e29b993e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44311 } } }
I20260809 16:38:53.023167 1812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:53.023479 1823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:53.023921 1823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:53.024019 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1681
I20260809 16:38:53.024137 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:53.024322 31807 external_mini_cluster.cc:1428] Setting key da598443250574b62b8f1a1ed7e4a4ed
I20260809 16:38:53.025413 1823 catalog_manager.cc:1347] Generated new cluster ID: 765b2fc023ab4cda886b791007f7b20f
I20260809 16:38:53.025465 1823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:53.027047 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:53.024758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:40669 (local address 127.0.0.1:44311)
0809 16:38:53.024906 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:53.024916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:53.025215 (+ 299us) server_negotiation.cc:407] Connection header received
0809 16:38:53.025339 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:53.025346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:53.025377 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:53.025524 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:53.026551 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:53.026555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:53.026555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:53.026560 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:53.026617 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:53.026633 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:53.026670 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:53.026797 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 16:38:53.026846 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260809 16:38:53.028252 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:53.030723 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:53.029715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.177:55931 (local address 127.0.0.1:44311)
0809 16:38:53.029746 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:53.029748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:53.029852 (+ 104us) server_negotiation.cc:407] Connection header received
0809 16:38:53.029894 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:53.029895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:53.029899 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:53.029966 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:53.030507 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:53.030508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:53.030509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:53.030513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:53.030529 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:53.030533 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:53.030607 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:53.030651 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 16:38:53.030664 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:38:53.050732 1823 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:53.050912 1823 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:53.059542 1823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ebecf8cbc984090b9a5207e29b993e6: Generated new TSK 0
I20260809 16:38:53.059725 1823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:53.064049 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1681
2026-08-09T16:38:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:53 dist-test-slave-21b9 krb5kdc[1859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:53 dist-test-slave-21b9 krb5kdc[1859](info): set up 2 sockets
Aug 09 16:38:53 dist-test-slave-21b9 krb5kdc[1859](info): commencing operation
krb5kdc: starting...
W20260809 16:38:54.222965 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[1859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:54.339126 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40117
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:41207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:40117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:54.420815 1884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:54.421025 1884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:54.421047 1884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:54.422127 1884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:54.422163 1884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:54.422174 1884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:54.422185 1884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:54.422196 1884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:54.423687 1884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:41207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:40117
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:54.423890 1884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:54.424070 1884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:54.428490 1891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:54.428501 1892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:54.428501 1895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:54.428700 1884 server_base.cc:1034] running on GCE node
I20260809 16:38:54.428956 1884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:54.429235 1884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:54.430398 1884 hybrid_clock.cc:648] HybridClock initialized: now 1786293534430385 us; error 31 us; skew 500 ppm
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[1859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293534, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:54.436957 1884 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:54.437394 1884 webserver.cc:460] Webserver started at http://127.31.15.254:44955/ using document root <none> and password file <none>
I20260809 16:38:54.437603 1884 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:54.437651 1884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:54.437773 1884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:54.438786 1884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5"
format_stamp: "Formatted at 2026-08-09 16:38:54 on dist-test-slave-21b9"
server_key: "b24ce6d979fbbfa7024ba4253c629a7e"
server_key_iv: "c46c1158611ad430c68ccd936508471e"
server_key_version: "encryptionkey@0"
I20260809 16:38:54.439114 1884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5"
format_stamp: "Formatted at 2026-08-09 16:38:54 on dist-test-slave-21b9"
server_key: "b24ce6d979fbbfa7024ba4253c629a7e"
server_key_iv: "c46c1158611ad430c68ccd936508471e"
server_key_version: "encryptionkey@0"
I20260809 16:38:54.440374 1884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 16:38:54.441051 1908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:54.441205 1884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:54.441287 1884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5"
format_stamp: "Formatted at 2026-08-09 16:38:54 on dist-test-slave-21b9"
server_key: "b24ce6d979fbbfa7024ba4253c629a7e"
server_key_iv: "c46c1158611ad430c68ccd936508471e"
server_key_version: "encryptionkey@0"
I20260809 16:38:54.441352 1884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:54.458189 1884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:54.460045 1884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:54.460186 1884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:54.464092 1884 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:40117
I20260809 16:38:54.464134 2012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:40117 every 8 connection(s)
I20260809 16:38:54.464466 1884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:54.465302 2013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:54.467630 2013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5: Bootstrap starting.
I20260809 16:38:54.468170 2013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:54.468415 2013 log.cc:824] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:54.469034 2013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5: No bootstrap required, opened a new log
I20260809 16:38:54.470697 2013 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } }
I20260809 16:38:54.470875 2013 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:54.470903 2013 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1e1adc963c4b8fa8a1610c8cc95ec5, State: Initialized, Role: FOLLOWER
I20260809 16:38:54.471024 2013 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } }
I20260809 16:38:54.471101 2013 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:54.471131 2013 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:54.471161 2013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:54.471985 2013 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } }
I20260809 16:38:54.472074 2013 leader_election.cc:302] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf1e1adc963c4b8fa8a1610c8cc95ec5; no voters:
I20260809 16:38:54.472189 2013 leader_election.cc:288] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:54.472245 2018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:54.472412 2013 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:54.472422 2018 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [term 1 LEADER]: Becoming Leader. State: Replica: cf1e1adc963c4b8fa8a1610c8cc95ec5, State: Running, Role: LEADER
I20260809 16:38:54.472585 2018 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } }
I20260809 16:38:54.472978 2020 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1e1adc963c4b8fa8a1610c8cc95ec5. Latest consensus state: current_term: 1 leader_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } } }
I20260809 16:38:54.473014 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1e1adc963c4b8fa8a1610c8cc95ec5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 40117 } } }
I20260809 16:38:54.473145 2020 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:54.473157 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:54.473500 2030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:54.474025 2030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:54.474083 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 1884
I20260809 16:38:54.474202 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:54.474375 31807 external_mini_cluster.cc:1428] Setting key 9866ccf353d1958d28618e0f1648b054
I20260809 16:38:54.475602 2030 catalog_manager.cc:1347] Generated new cluster ID: 300014c2a58449dbb75d50a780c7b1a0
I20260809 16:38:54.475658 2030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[1859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:54.484063 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:54.475025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52419 (local address 127.31.15.254:40117)
0809 16:38:54.475656 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:54.475666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:54.475698 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:38:54.475805 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:54.475810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:54.475851 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:54.475967 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:54.476592 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:54.477486 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:54.478248 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:54.478700 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:54.478795 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:54.481020 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:54.481024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:54.481024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:54.481037 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:54.482552 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:54.482910 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:54.482913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:54.482913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:54.483012 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:54.483214 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:54.483215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:54.483216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:54.483468 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:54.483635 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:54.483758 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:38:54.483856 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":71,"threads_started":1}
I20260809 16:38:54.484946 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:54.491643 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:54.486156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32873 (local address 127.31.15.254:40117)
0809 16:38:54.486189 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:54.486191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:54.486298 (+ 107us) server_negotiation.cc:407] Connection header received
0809 16:38:54.486348 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:54.486349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:54.486358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:54.486401 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:54.486856 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:54.487381 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:54.488090 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:54.488289 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:54.488341 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:54.489399 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:54.489400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:54.489401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:54.489407 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:54.490608 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:54.490925 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:54.490926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:54.490926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:54.490978 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:54.491163 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:54.491163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:54.491164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:54.491263 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:54.491381 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:54.491508 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 16:38:54.491565 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 16:38:54.497711 2030 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:54.498181 2030 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:54.508334 2030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf1e1adc963c4b8fa8a1610c8cc95ec5: Generated new TSK 0
I20260809 16:38:54.508496 2030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:54.512463 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 1884
W20260809 16:38:54.526224 1857 connection.cc:535] client connection to 127.31.15.254:40117 recv error: Network error: failed to read from TLS socket (remote: 127.31.15.254:40117): Connection reset by peer (error 104)
2026-08-09T16:38:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[2071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[2071](info): set up 2 sockets
Aug 09 16:38:54 dist-test-slave-21b9 krb5kdc[2071](info): commencing operation
krb5kdc: starting...
W20260809 16:38:55.664796 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:55 dist-test-slave-21b9 krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:55.776096 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:33263
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:36273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:33263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:55.855727 2091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:55.855914 2091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:55.855945 2091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:55.857046 2091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:55.857086 2091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:55.857106 2091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:55.857121 2091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:55.857138 2091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:55.858544 2091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:36273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:33263
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:55.858779 2091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:55.858989 2091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:55.863498 2102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:55.863534 2091 server_base.cc:1034] running on GCE node
W20260809 16:38:55.863596 2104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:55.863504 2100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:55.863935 2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:55.864169 2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:55.865309 2091 hybrid_clock.cc:648] HybridClock initialized: now 1786293535865291 us; error 33 us; skew 500 ppm
Aug 09 16:38:55 dist-test-slave-21b9 krb5kdc[2071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293535, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:55.867645 2091 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:55.868063 2091 webserver.cc:460] Webserver started at http://127.31.15.254:38647/ using document root <none> and password file <none>
I20260809 16:38:55.868235 2091 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:55.868274 2091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:55.868362 2091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:55.869328 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "a43193ee39344432ac45b3b4a430845a"
format_stamp: "Formatted at 2026-08-09 16:38:55 on dist-test-slave-21b9"
server_key: "e594cca33f15ee6d193f6a6167ce3e08"
server_key_iv: "1297e3f6c43e1232b016e7775eea7c4c"
server_key_version: "encryptionkey@0"
I20260809 16:38:55.869639 2091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "a43193ee39344432ac45b3b4a430845a"
format_stamp: "Formatted at 2026-08-09 16:38:55 on dist-test-slave-21b9"
server_key: "e594cca33f15ee6d193f6a6167ce3e08"
server_key_iv: "1297e3f6c43e1232b016e7775eea7c4c"
server_key_version: "encryptionkey@0"
I20260809 16:38:55.870795 2091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:55.871429 2112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:55.871584 2091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:55.871665 2091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "a43193ee39344432ac45b3b4a430845a"
format_stamp: "Formatted at 2026-08-09 16:38:55 on dist-test-slave-21b9"
server_key: "e594cca33f15ee6d193f6a6167ce3e08"
server_key_iv: "1297e3f6c43e1232b016e7775eea7c4c"
server_key_version: "encryptionkey@0"
I20260809 16:38:55.871719 2091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:55.885967 2091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:55.887907 2091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:55.888011 2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:55.891644 2091 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:33263
I20260809 16:38:55.891688 2202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:33263 every 8 connection(s)
I20260809 16:38:55.891969 2091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:55.892764 2203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:55.894919 2203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a: Bootstrap starting.
I20260809 16:38:55.895475 2203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:55.895746 2203 log.cc:824] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:55.896385 2203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a: No bootstrap required, opened a new log
I20260809 16:38:55.897918 2203 raft_consensus.cc:348] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } }
I20260809 16:38:55.898048 2203 raft_consensus.cc:374] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:55.898069 2203 raft_consensus.cc:729] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43193ee39344432ac45b3b4a430845a, State: Initialized, Role: FOLLOWER
I20260809 16:38:55.898204 2203 consensus_queue.cc:260] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } }
I20260809 16:38:55.898285 2203 raft_consensus.cc:388] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:55.898307 2203 raft_consensus.cc:482] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:55.898336 2203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:55.899128 2203 raft_consensus.cc:504] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } }
I20260809 16:38:55.899219 2203 leader_election.cc:302] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a43193ee39344432ac45b3b4a430845a; no voters:
I20260809 16:38:55.899389 2203 leader_election.cc:288] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:55.899447 2206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:55.899629 2203 sys_catalog.cc:564] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:55.899637 2206 raft_consensus.cc:686] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [term 1 LEADER]: Becoming Leader. State: Replica: a43193ee39344432ac45b3b4a430845a, State: Running, Role: LEADER
I20260809 16:38:55.899830 2206 consensus_queue.cc:237] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } }
I20260809 16:38:55.900199 2208 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a43193ee39344432ac45b3b4a430845a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } } }
I20260809 16:38:55.900297 2208 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:55.900404 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2091
I20260809 16:38:55.900506 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:55.900558 2219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:55.900187 2209 sys_catalog.cc:455] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a43193ee39344432ac45b3b4a430845a. Latest consensus state: current_term: 1 leader_uuid: "a43193ee39344432ac45b3b4a430845a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a43193ee39344432ac45b3b4a430845a" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 33263 } } }
I20260809 16:38:55.900719 31807 external_mini_cluster.cc:1428] Setting key cfbee689153fc4473315404b4de41422
I20260809 16:38:55.900803 2209 sys_catalog.cc:458] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:55.901069 2219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:55.904700 2219 catalog_manager.cc:1347] Generated new cluster ID: 2ff31a4889e447a8b1f59d1f359c7c2c
I20260809 16:38:55.904760 2219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:55.906471 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:55.901129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43309 (local address 127.31.15.254:33263)
0809 16:38:55.901450 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:55.901459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:55.901490 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:38:55.901750 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:55.901758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:55.901816 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:55.901960 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:55.902184 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:55.903533 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:55.904640 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:55.905076 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:55.905181 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:55.905870 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:55.905883 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:55.905883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:55.906049 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:55.906081 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:55.906093 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:55.906174 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:55.906275 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 16:38:55.906303 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":74,"threads_started":1}
I20260809 16:38:55.907433 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:55.917610 2219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:55.918249 2219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:55.918500 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:55.908829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57597 (local address 127.31.15.254:33263)
0809 16:38:55.908865 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:55.908867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:55.915458 (+ 6591us) server_negotiation.cc:407] Connection header received
0809 16:38:55.915527 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:55.915528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:55.915540 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:55.915594 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:55.915749 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:55.916419 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:55.917159 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:55.917485 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:55.917556 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:55.918191 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:55.918193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:55.918193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:55.918198 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:55.918213 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:55.918217 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:55.918275 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:55.918409 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 16:38:55.918422 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":171,"server-negotiator.queue_time_us":19}
I20260809 16:38:55.923950 2219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a43193ee39344432ac45b3b4a430845a: Generated new TSK 0
I20260809 16:38:55.924129 2219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:55.926163 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2091
W20260809 16:38:55.938136 2065 connection.cc:535] client connection to 127.31.15.254:33263 recv error: Network error: failed to read from TLS socket (remote: 127.31.15.254:33263): Connection reset by peer (error 104)
2026-08-09T16:38:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:55 dist-test-slave-21b9 krb5kdc[2252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:55 dist-test-slave-21b9 krb5kdc[2252](info): set up 2 sockets
Aug 09 16:38:55 dist-test-slave-21b9 krb5kdc[2252](info): commencing operation
krb5kdc: starting...
W20260809 16:38:57.079028 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:57 dist-test-slave-21b9 krb5kdc[2252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:57.194802 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39615
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:43155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:57.274686 2270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:57.274878 2270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:57.274909 2270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:57.276098 2270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:57.276144 2270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:57.276163 2270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:57.276180 2270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:57.276196 2270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:57.277685 2270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:43155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:39615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:57.277912 2270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:57.278118 2270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:57.282665 2279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:57.282680 2283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:57.282702 2280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:57.282786 2270 server_base.cc:1034] running on GCE node
I20260809 16:38:57.283097 2270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:57.283409 2270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:57.284559 2270 hybrid_clock.cc:648] HybridClock initialized: now 1786293537284542 us; error 33 us; skew 500 ppm
Aug 09 16:38:57 dist-test-slave-21b9 krb5kdc[2252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293537, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:57.286952 2270 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:57.287389 2270 webserver.cc:460] Webserver started at http://127.31.15.254:46215/ using document root <none> and password file <none>
I20260809 16:38:57.287617 2270 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:57.287667 2270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:57.287791 2270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:57.288812 2270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2"
format_stamp: "Formatted at 2026-08-09 16:38:57 on dist-test-slave-21b9"
server_key: "87d28610246cfd65c5b837bb50cfc96c"
server_key_iv: "eaa0ec4e3eec8443d9fdd359fef36459"
server_key_version: "encryptionkey@0"
I20260809 16:38:57.289147 2270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2"
format_stamp: "Formatted at 2026-08-09 16:38:57 on dist-test-slave-21b9"
server_key: "87d28610246cfd65c5b837bb50cfc96c"
server_key_iv: "eaa0ec4e3eec8443d9fdd359fef36459"
server_key_version: "encryptionkey@0"
I20260809 16:38:57.290355 2270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:38:57.291018 2300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:57.291174 2270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:57.291257 2270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2"
format_stamp: "Formatted at 2026-08-09 16:38:57 on dist-test-slave-21b9"
server_key: "87d28610246cfd65c5b837bb50cfc96c"
server_key_iv: "eaa0ec4e3eec8443d9fdd359fef36459"
server_key_version: "encryptionkey@0"
I20260809 16:38:57.291347 2270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:57.294443 2270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:57.296756 2270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:57.296890 2270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:57.300721 2270 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:39615
I20260809 16:38:57.300753 2384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:39615 every 8 connection(s)
I20260809 16:38:57.301036 2270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:57.301981 2386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:57.304113 2386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2: Bootstrap starting.
I20260809 16:38:57.304638 2386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:57.304898 2386 log.cc:824] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:57.305509 2386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2: No bootstrap required, opened a new log
I20260809 16:38:57.307050 2386 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } }
I20260809 16:38:57.307176 2386 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:57.307209 2386 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3a0edf88e24f5b988fbe5ccc28cce2, State: Initialized, Role: FOLLOWER
I20260809 16:38:57.307368 2386 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } }
I20260809 16:38:57.307435 2386 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:57.307471 2386 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:57.307516 2386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:57.308272 2386 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } }
I20260809 16:38:57.308357 2386 leader_election.cc:302] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3a0edf88e24f5b988fbe5ccc28cce2; no voters:
I20260809 16:38:57.308497 2386 leader_election.cc:288] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:57.308540 2390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:57.308673 2390 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [term 1 LEADER]: Becoming Leader. State: Replica: fc3a0edf88e24f5b988fbe5ccc28cce2, State: Running, Role: LEADER
I20260809 16:38:57.308730 2386 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:57.308760 2390 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } }
I20260809 16:38:57.309170 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2270
I20260809 16:38:57.309303 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:57.309536 31807 external_mini_cluster.cc:1428] Setting key adf8ac3a0e46d74fef921d917ae5e346
I20260809 16:38:57.309778 2393 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3a0edf88e24f5b988fbe5ccc28cce2. Latest consensus state: current_term: 1 leader_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } } }
I20260809 16:38:57.309850 2393 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:57.310089 2392 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a0edf88e24f5b988fbe5ccc28cce2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 39615 } } }
I20260809 16:38:57.310222 2411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:57.310428 2392 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:57.310897 2411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:57.311900 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:57.310131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55651 (local address 127.31.15.254:39615)
0809 16:38:57.310290 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:57.310299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:57.310333 (+ 34us) server_negotiation.cc:407] Connection header received
0809 16:38:57.310449 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:57.310455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:57.310485 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:57.310613 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:57.311214 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:57.311217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:57.311218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:57.311222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:57.311261 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:57.311299 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:57.311354 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:57.311425 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 16:38:57.311759 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260809 16:38:57.312333 2411 catalog_manager.cc:1347] Generated new cluster ID: 2c87e72e9f624f55b064f61d2643d9d5
I20260809 16:38:57.312383 2411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:38:57.312788 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:57.315114 2416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:57.314261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40907 (local address 127.31.15.254:39615)
0809 16:38:57.314296 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:57.314297 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:57.314304 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:38:57.314333 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:57.314333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:57.314337 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:57.314378 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:57.314898 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:57.314900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:57.314900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:38:57.314905 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:57.314928 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:38:57.314932 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:57.314992 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:57.315032 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 16:38:57.315044 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:38:57.332479 2411 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:57.332692 2411 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:57.334918 2411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3a0edf88e24f5b988fbe5ccc28cce2: Generated new TSK 0
I20260809 16:38:57.335090 2411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:57.341712 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2270
2026-08-09T16:38:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:57 dist-test-slave-21b9 krb5kdc[2433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:57 dist-test-slave-21b9 krb5kdc[2433](info): set up 2 sockets
Aug 09 16:38:57 dist-test-slave-21b9 krb5kdc[2433](info): commencing operation
krb5kdc: starting...
W20260809 16:38:58.502007 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:38:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:38:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:38:58.615927 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34271
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:32817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:34271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:38:58.695143 2452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:38:58.695384 2452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:38:58.695421 2452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:38:58.696535 2452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:38:58.696586 2452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:38:58.696606 2452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:38:58.696621 2452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:38:58.696637 2452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:38:58.698066 2452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:32817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:34271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:38:58.698308 2452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:38:58.698508 2452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:38:58.703142 2464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:58.703181 2469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:38:58.703142 2474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:38:58.703227 2452 server_base.cc:1034] running on GCE node
I20260809 16:38:58.703631 2452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:38:58.703881 2452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:38:58.705031 2452 hybrid_clock.cc:648] HybridClock initialized: now 1786293538705013 us; error 29 us; skew 500 ppm
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293538, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:38:58.707376 2452 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:38:58.707787 2452 webserver.cc:460] Webserver started at http://127.31.15.254:41991/ using document root <none> and password file <none>
I20260809 16:38:58.707983 2452 fs_manager.cc:359] Metadata directory not provided
I20260809 16:38:58.708030 2452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:38:58.708150 2452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:38:58.709165 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "9cb26db93fa2474f91a36fdab062e1d5"
format_stamp: "Formatted at 2026-08-09 16:38:58 on dist-test-slave-21b9"
server_key: "7877ad9ec834ef64ad8286092f8b1708"
server_key_iv: "e9408b6042e71ddf16cb93639f613c61"
server_key_version: "encryptionkey@0"
I20260809 16:38:58.709498 2452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "9cb26db93fa2474f91a36fdab062e1d5"
format_stamp: "Formatted at 2026-08-09 16:38:58 on dist-test-slave-21b9"
server_key: "7877ad9ec834ef64ad8286092f8b1708"
server_key_iv: "e9408b6042e71ddf16cb93639f613c61"
server_key_version: "encryptionkey@0"
I20260809 16:38:58.710745 2452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 16:38:58.711410 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:38:58.711565 2452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:38:58.711649 2452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "9cb26db93fa2474f91a36fdab062e1d5"
format_stamp: "Formatted at 2026-08-09 16:38:58 on dist-test-slave-21b9"
server_key: "7877ad9ec834ef64ad8286092f8b1708"
server_key_iv: "e9408b6042e71ddf16cb93639f613c61"
server_key_version: "encryptionkey@0"
I20260809 16:38:58.711715 2452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:38:58.728858 2452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:38:58.730528 2452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:38:58.730657 2452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:38:58.734421 2452 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:34271
I20260809 16:38:58.734462 2560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:34271 every 8 connection(s)
I20260809 16:38:58.734777 2452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:38:58.735651 2562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:38:58.737814 2562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5: Bootstrap starting.
I20260809 16:38:58.738426 2562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5: Neither blocks nor log segments found. Creating new log.
I20260809 16:38:58.738727 2562 log.cc:824] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5: Log is configured to *not* fsync() on all Append() calls
I20260809 16:38:58.739413 2562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5: No bootstrap required, opened a new log
I20260809 16:38:58.740221 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2452
I20260809 16:38:58.740341 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:38:58.740512 31807 external_mini_cluster.cc:1428] Setting key 525d87b4e21ec54e87a8ac2305a13d22
I20260809 16:38:58.741202 2562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } }
I20260809 16:38:58.741341 2562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:38:58.741374 2562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb26db93fa2474f91a36fdab062e1d5, State: Initialized, Role: FOLLOWER
I20260809 16:38:58.741509 2562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } }
I20260809 16:38:58.741595 2562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:38:58.741627 2562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:38:58.741674 2562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:38:58.742579 2562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } }
I20260809 16:38:58.742668 2562 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb26db93fa2474f91a36fdab062e1d5; no voters:
I20260809 16:38:58.742817 2562 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:38:58.742857 2568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:38:58.742990 2568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb26db93fa2474f91a36fdab062e1d5, State: Running, Role: LEADER
I20260809 16:38:58.743060 2562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:38:58.743081 2568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } }
I20260809 16:38:58.743429 2568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb26db93fa2474f91a36fdab062e1d5. Latest consensus state: current_term: 1 leader_uuid: "9cb26db93fa2474f91a36fdab062e1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } } }
I20260809 16:38:58.743505 2568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:58.743672 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb26db93fa2474f91a36fdab062e1d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb26db93fa2474f91a36fdab062e1d5" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34271 } } }
I20260809 16:38:58.743736 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5 [sys.catalog]: This master's current role is: LEADER
I20260809 16:38:58.743837 2579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:38:58.744544 2579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:38:58.746196 2579 catalog_manager.cc:1347] Generated new cluster ID: 138d2bfba23948618458bd719c0687d3
I20260809 16:38:58.746249 2579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:38:58.752562 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:58.740980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36769 (local address 127.31.15.254:34271)
0809 16:38:58.741164 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:58.741172 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:58.741197 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:38:58.741288 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:58.741292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:58.741339 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:58.741459 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:58.742216 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:58.743436 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:58.744483 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:58.745051 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:58.745170 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:58.748008 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:58.748011 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:58.748012 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:58.748022 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:58.749803 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:58.750191 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:58.750193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:58.750193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:58.750318 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:58.750538 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:58.750538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:58.750539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:58.750801 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:58.750946 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:58.752220 (+ 1274us) server_negotiation.cc:299] Negotiation successful
0809 16:38:58.752351 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
I20260809 16:38:58.753546 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:38:58.761615 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:38:58.754945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54521 (local address 127.31.15.254:34271)
0809 16:38:58.754980 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 16:38:58.754982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:38:58.755150 (+ 168us) server_negotiation.cc:407] Connection header received
0809 16:38:58.755223 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:38:58.755224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:38:58.755233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:38:58.755256 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:38:58.755893 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:58.756618 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:58.757555 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:38:58.757868 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:38:58.757899 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:38:58.759137 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:38:58.759138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:38:58.759139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:38:58.759145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:38:58.760606 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:58.760930 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:58.760931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:58.760932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:58.760974 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:38:58.761130 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:38:58.761130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:38:58.761130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:38:58.761232 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:38:58.761290 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 16:38:58.761457 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 16:38:58.761521 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:38:58.771371 2579 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:38:58.772102 2579 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:38:58.779461 2579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb26db93fa2474f91a36fdab062e1d5: Generated new TSK 0
I20260809 16:38:58.779629 2579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:38:58.785653 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2452
2026-08-09T16:38:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2616](info): set up 2 sockets
Aug 09 16:38:58 dist-test-slave-21b9 krb5kdc[2616](info): commencing operation
krb5kdc: starting...
W20260809 16:38:59.938246 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:39:00 dist-test-slave-21b9 krb5kdc[2616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:39:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:39:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:39:00.054194 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:44681
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:44681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:39:00.134485 2639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:00.134676 2639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:00.134706 2639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:00.135813 2639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:00.135854 2639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:00.135872 2639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:00.135888 2639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:00.135902 2639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:00.137367 2639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:44681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:44681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:00.137605 2639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:00.137813 2639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:00.142599 2644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:00.142635 2645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:00.142669 2639 server_base.cc:1034] running on GCE node
W20260809 16:39:00.142637 2647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:00.143008 2639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:00.143252 2639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:00.144418 2639 hybrid_clock.cc:648] HybridClock initialized: now 1786293540144402 us; error 36 us; skew 500 ppm
Aug 09 16:39:00 dist-test-slave-21b9 krb5kdc[2616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293540, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:00.151865 2639 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:39:00.152308 2639 webserver.cc:460] Webserver started at http://127.31.15.254:40631/ using document root <none> and password file <none>
I20260809 16:39:00.152534 2639 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:00.152581 2639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:00.152706 2639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:00.153733 2639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "be7abbb5c4e34da19edaf58eda05db1f"
format_stamp: "Formatted at 2026-08-09 16:39:00 on dist-test-slave-21b9"
server_key: "5d800478c59244d5952dfbc94a3b49ef"
server_key_iv: "b04de3becd322884b035831379db664f"
server_key_version: "encryptionkey@0"
I20260809 16:39:00.154064 2639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "be7abbb5c4e34da19edaf58eda05db1f"
format_stamp: "Formatted at 2026-08-09 16:39:00 on dist-test-slave-21b9"
server_key: "5d800478c59244d5952dfbc94a3b49ef"
server_key_iv: "b04de3becd322884b035831379db664f"
server_key_version: "encryptionkey@0"
I20260809 16:39:00.155288 2639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:39:00.155992 2657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:00.156138 2639 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:39:00.156225 2639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "be7abbb5c4e34da19edaf58eda05db1f"
format_stamp: "Formatted at 2026-08-09 16:39:00 on dist-test-slave-21b9"
server_key: "5d800478c59244d5952dfbc94a3b49ef"
server_key_iv: "b04de3becd322884b035831379db664f"
server_key_version: "encryptionkey@0"
I20260809 16:39:00.156291 2639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:00.167347 2639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:00.169234 2639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:00.169368 2639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:00.173303 2639 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:44681
I20260809 16:39:00.173338 2740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:44681 every 8 connection(s)
I20260809 16:39:00.173658 2639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:39:00.174480 2741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:00.176903 2741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f: Bootstrap starting.
I20260809 16:39:00.177479 2741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:00.177742 2741 log.cc:824] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:00.178514 2741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f: No bootstrap required, opened a new log
I20260809 16:39:00.179229 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2639
I20260809 16:39:00.179370 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:39:00.179559 31807 external_mini_cluster.cc:1428] Setting key 77aa2e52efb86effbf07d1e3601163c5
I20260809 16:39:00.180866 2741 raft_consensus.cc:348] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } }
I20260809 16:39:00.181041 2741 raft_consensus.cc:374] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:39:00.181102 2741 raft_consensus.cc:729] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be7abbb5c4e34da19edaf58eda05db1f, State: Initialized, Role: FOLLOWER
I20260809 16:39:00.181212 2741 consensus_queue.cc:260] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } }
I20260809 16:39:00.181289 2741 raft_consensus.cc:388] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:39:00.181320 2741 raft_consensus.cc:482] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:39:00.181362 2741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:00.182042 2741 raft_consensus.cc:504] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } }
I20260809 16:39:00.182166 2741 leader_election.cc:302] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be7abbb5c4e34da19edaf58eda05db1f; no voters:
I20260809 16:39:00.182320 2741 leader_election.cc:288] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:39:00.182370 2751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:39:00.182525 2751 raft_consensus.cc:686] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [term 1 LEADER]: Becoming Leader. State: Replica: be7abbb5c4e34da19edaf58eda05db1f, State: Running, Role: LEADER
I20260809 16:39:00.182551 2741 sys_catalog.cc:564] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:00.182621 2751 consensus_queue.cc:237] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } }
I20260809 16:39:00.183000 2755 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader be7abbb5c4e34da19edaf58eda05db1f. Latest consensus state: current_term: 1 leader_uuid: "be7abbb5c4e34da19edaf58eda05db1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } } }
I20260809 16:39:00.183106 2755 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:00.183012 2753 sys_catalog.cc:455] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be7abbb5c4e34da19edaf58eda05db1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be7abbb5c4e34da19edaf58eda05db1f" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 44681 } } }
I20260809 16:39:00.183183 2753 sys_catalog.cc:458] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:00.183323 2765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:39:00.183816 2765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:39:00.185390 2765 catalog_manager.cc:1347] Generated new cluster ID: fe86b67c588943f4aef60359d36d8852
I20260809 16:39:00.185438 2765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:39:00.186214 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:00.180078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59093 (local address 127.31.15.254:44681)
0809 16:39:00.180241 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:00.180250 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:00.180283 (+ 33us) server_negotiation.cc:407] Connection header received
0809 16:39:00.180823 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:00.180833 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:00.180896 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:00.181052 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:00.181295 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:00.182931 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:00.184095 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:00.184723 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:00.184831 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:00.185523 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:00.185526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:00.185527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:39:00.185734 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:00.185775 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:39:00.185790 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:00.185856 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:00.185931 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 16:39:00.185992 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260809 16:39:00.187036 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:39:00.191744 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:00.188433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52135 (local address 127.31.15.254:44681)
0809 16:39:00.188471 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:00.188472 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:00.188653 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:39:00.188678 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:00.188679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:00.188690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:00.188752 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:00.188848 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:00.189560 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:00.190477 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:00.190777 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:00.190848 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:00.191529 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:00.191530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:00.191531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:39:00.191535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:00.191549 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:39:00.191553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:00.191609 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:00.191650 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:39:00.191672 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:39:00.196712 2765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:39:00.197170 2765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:39:00.203164 2765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be7abbb5c4e34da19edaf58eda05db1f: Generated new TSK 0
I20260809 16:39:00.203341 2765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:39:00.207376 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2639
2026-08-09T16:39:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:39:00 dist-test-slave-21b9 krb5kdc[2809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:39:00 dist-test-slave-21b9 krb5kdc[2809](info): set up 2 sockets
Aug 09 16:39:00 dist-test-slave-21b9 krb5kdc[2809](info): commencing operation
krb5kdc: starting...
W20260809 16:39:01.381373 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:39:01 dist-test-slave-21b9 krb5kdc[2809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:39:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:39:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:39:01.495879 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:42629
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:42629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:39:01.576073 2832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:01.576308 2832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:01.576335 2832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:01.577476 2832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:01.577518 2832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:01.577533 2832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:01.577544 2832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:01.577554 2832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:01.578984 2832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:42629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:42629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:01.579181 2832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:01.579396 2832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:01.583891 2845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:01.583984 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:01.584082 2842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:01.584244 2832 server_base.cc:1034] running on GCE node
I20260809 16:39:01.584481 2832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:01.584745 2832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:01.585875 2832 hybrid_clock.cc:648] HybridClock initialized: now 1786293541585853 us; error 36 us; skew 500 ppm
Aug 09 16:39:01 dist-test-slave-21b9 krb5kdc[2809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293541, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:01.588219 2832 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:39:01.588634 2832 webserver.cc:460] Webserver started at http://127.31.15.254:38259/ using document root <none> and password file <none>
I20260809 16:39:01.588814 2832 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:01.588853 2832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:01.588944 2832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:01.589910 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "16033900d7d94b7c98cabd730b003896"
format_stamp: "Formatted at 2026-08-09 16:39:01 on dist-test-slave-21b9"
server_key: "6e94deb5def9d50233613f7a8e76750f"
server_key_iv: "add798a4e49305f1c9d04f627ded42af"
server_key_version: "encryptionkey@0"
I20260809 16:39:01.590197 2832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "16033900d7d94b7c98cabd730b003896"
format_stamp: "Formatted at 2026-08-09 16:39:01 on dist-test-slave-21b9"
server_key: "6e94deb5def9d50233613f7a8e76750f"
server_key_iv: "add798a4e49305f1c9d04f627ded42af"
server_key_version: "encryptionkey@0"
I20260809 16:39:01.591387 2832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 16:39:01.592087 2857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:01.592257 2832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:01.592339 2832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "16033900d7d94b7c98cabd730b003896"
format_stamp: "Formatted at 2026-08-09 16:39:01 on dist-test-slave-21b9"
server_key: "6e94deb5def9d50233613f7a8e76750f"
server_key_iv: "add798a4e49305f1c9d04f627ded42af"
server_key_version: "encryptionkey@0"
I20260809 16:39:01.592393 2832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:01.595498 2832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:01.597189 2832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:01.597288 2832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:01.600992 2832 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:42629
I20260809 16:39:01.601028 2929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:42629 every 8 connection(s)
I20260809 16:39:01.601312 2832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:39:01.602250 2935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:01.604511 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896: Bootstrap starting.
I20260809 16:39:01.605039 2935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:01.605283 2935 log.cc:824] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:01.605882 2935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896: No bootstrap required, opened a new log
I20260809 16:39:01.607728 2935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } }
I20260809 16:39:01.607860 2935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 16:39:01.607892 2935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16033900d7d94b7c98cabd730b003896, State: Initialized, Role: FOLLOWER
I20260809 16:39:01.608045 2935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } }
I20260809 16:39:01.608132 2935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 16:39:01.608171 2935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 16:39:01.608222 2935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:01.609046 2935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } }
I20260809 16:39:01.609149 2935 leader_election.cc:302] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16033900d7d94b7c98cabd730b003896; no voters:
I20260809 16:39:01.609311 2935 leader_election.cc:288] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 16:39:01.609349 2941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:39:01.609488 2941 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [term 1 LEADER]: Becoming Leader. State: Replica: 16033900d7d94b7c98cabd730b003896, State: Running, Role: LEADER
I20260809 16:39:01.609545 2935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:01.609578 2941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } }
I20260809 16:39:01.609922 2943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16033900d7d94b7c98cabd730b003896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } } }
I20260809 16:39:01.610023 2943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:01.610178 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 2832
I20260809 16:39:01.610298 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:39:01.610491 31807 external_mini_cluster.cc:1428] Setting key 44bef49ff4d3ff28194b1550a45c5f25
I20260809 16:39:01.610708 2958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:39:01.609925 2944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16033900d7d94b7c98cabd730b003896. Latest consensus state: current_term: 1 leader_uuid: "16033900d7d94b7c98cabd730b003896" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16033900d7d94b7c98cabd730b003896" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 42629 } } }
I20260809 16:39:01.611344 2944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:01.611459 2958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:39:01.612840 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:01.611142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52903 (local address 127.31.15.254:42629)
0809 16:39:01.611309 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:01.611319 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:01.611442 (+ 123us) server_negotiation.cc:407] Connection header received
0809 16:39:01.611554 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:01.611571 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:01.611602 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:01.611725 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:01.612355 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:01.612359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:01.612359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:39:01.612363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:01.612400 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:39:01.612411 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:01.612466 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:01.612652 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 16:39:01.612696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260809 16:39:01.613142 2958 catalog_manager.cc:1347] Generated new cluster ID: 6dd1a8babf8d4faea7ce290d732d2c80
I20260809 16:39:01.613193 2958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:39:01.615803 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:39:01.618049 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:01.617087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54615 (local address 127.31.15.254:42629)
0809 16:39:01.617126 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:01.617128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:01.617233 (+ 105us) server_negotiation.cc:407] Connection header received
0809 16:39:01.617287 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:01.617288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:01.617292 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:01.617337 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:01.617849 (+ 512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:01.617850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:01.617850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 16:39:01.617854 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:01.617868 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 16:39:01.617873 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:01.617918 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:01.617966 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 16:39:01.617977 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:39:01.620293 2958 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:39:01.620436 2958 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:39:01.628224 2958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16033900d7d94b7c98cabd730b003896: Generated new TSK 0
I20260809 16:39:01.628382 2958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:39:01.633073 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 2832
2026-08-09T16:39:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1419 ms)
[----------] 18 tests from AuthTokenIssuingTest (25784 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:39:01 dist-test-slave-21b9 krb5kdc[2992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:39:01 dist-test-slave-21b9 krb5kdc[2992](info): set up 2 sockets
Aug 09 16:39:01 dist-test-slave-21b9 krb5kdc[2992](info): commencing operation
krb5kdc: starting...
W20260809 16:39:02.794762 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:39:02 dist-test-slave-21b9 krb5kdc[2992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:39:02.909175 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34569
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:39:02.988485 3011 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:02.988698 3011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:02.988723 3011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:02.988746 3011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:02.989843 3011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:02.989882 3011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:02.989902 3011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:02.989918 3011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:02.989935 3011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:02.991430 3011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:34569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:02.991678 3011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:02.991873 3011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:02.996424 3019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:02.996435 3024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:02.996485 3022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:02.996572 3011 server_base.cc:1034] running on GCE node
I20260809 16:39:02.996873 3011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:02.997139 3011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:02.998287 3011 hybrid_clock.cc:648] HybridClock initialized: now 1786293542998270 us; error 34 us; skew 500 ppm
Aug 09 16:39:02 dist-test-slave-21b9 krb5kdc[2992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:03.000658 3011 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:39:03.001067 3011 webserver.cc:460] Webserver started at http://127.31.15.254:45409/ using document root <none> and password file <none>
I20260809 16:39:03.001257 3011 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:03.001304 3011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:03.001430 3011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:03.002408 3011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "877a763676df45988bea8851f8d6fe39"
format_stamp: "Formatted at 2026-08-09 16:39:02 on dist-test-slave-21b9"
server_key: "7550c7ea4ac45984e49a6a767f19ba34"
server_key_iv: "1502d2b50abc5d1802ad5ffaa398b75b"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.002736 3011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "877a763676df45988bea8851f8d6fe39"
format_stamp: "Formatted at 2026-08-09 16:39:02 on dist-test-slave-21b9"
server_key: "7550c7ea4ac45984e49a6a767f19ba34"
server_key_iv: "1502d2b50abc5d1802ad5ffaa398b75b"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.004016 3011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:39:03.004654 3032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:03.004802 3011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:03.004885 3011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "877a763676df45988bea8851f8d6fe39"
format_stamp: "Formatted at 2026-08-09 16:39:02 on dist-test-slave-21b9"
server_key: "7550c7ea4ac45984e49a6a767f19ba34"
server_key_iv: "1502d2b50abc5d1802ad5ffaa398b75b"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.004944 3011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:03.013707 3011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:03.015314 3011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:03.015448 3011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:03.019155 3011 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:34569
I20260809 16:39:03.019187 3130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:34569 every 8 connection(s)
I20260809 16:39:03.019522 3011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:39:03.020454 3131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:03.021919 3131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.023236 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3011
I20260809 16:39:03.023383 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:39:03.023561 31807 external_mini_cluster.cc:1428] Setting key 5f7aedc060ee73aeceb0405c5533901e
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:39:03.031203 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.022537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34569 (local address 127.0.0.1:48266)
0809 16:39:03.022692 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.022724 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.023227 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.023514 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.023517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.023796 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.023945 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.023949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.024928 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.024929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.025576 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.025578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.026052 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.026053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.026115 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.026571 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.026580 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.028043 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.029867 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.029874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.029875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.030063 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.030299 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.030300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.030300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.030368 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.030760 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.030763 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.030816 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.030878 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.030995 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260809 16:39:03.031211 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.022569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.31.15.254:34569)
0809 16:39:03.022731 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.022735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.022803 (+ 68us) server_negotiation.cc:407] Connection header received
0809 16:39:03.023389 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.023392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.023412 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.023530 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.024044 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.024841 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.025657 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.025954 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.026050 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.028136 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.028146 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.028146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.028154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.029789 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.030134 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.030137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.030138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.030239 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.030421 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.030421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.030422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.030694 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.030778 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.030930 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.031077 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260809 16:39:03.031975 3131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.15.253@KRBTEST.COM; defaulting to no policy
I20260809 16:39:03.032277 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.032175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60382)
0809 16:39:03.032216 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.032258 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 16:39:03.032388 3035 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
I20260809 16:39:03.032749 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.032670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60384)
0809 16:39:03.032704 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.032728 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:39:03.032835 3131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:43079: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
I20260809 16:39:03.048074 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:43079
--webserver_interface=127.31.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:39:03.082047 3131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } attempt: 1
I20260809 16:39:03.082563 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.082424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60394)
0809 16:39:03.082484 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.082533 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:39:03.082904 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.082831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60408)
0809 16:39:03.082866 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.082889 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:39:03.083004 3131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:43079: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
W20260809 16:39:03.130276 3141 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:03.130487 3141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:03.130502 3141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:03.130520 3141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:03.131639 3141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:03.131678 3141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:03.131691 3141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:03.131714 3141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:03.131736 3141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:03.133258 3141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:43079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:03.133487 3141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:03.133697 3141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:03.138264 3150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:03.138303 3151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:03.138476 3141 server_base.cc:1034] running on GCE node
W20260809 16:39:03.138271 3153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:03.138754 3141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:03.139045 3141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:03.140192 3141 hybrid_clock.cc:648] HybridClock initialized: now 1786293543140173 us; error 32 us; skew 500 ppm
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:03.142585 3141 init.cc:377] Logged in from keytab as kudu/127.31.15.253@KRBTEST.COM (short username kudu)
I20260809 16:39:03.143013 3141 webserver.cc:460] Webserver started at http://127.31.15.253:37051/ using document root <none> and password file <none>
I20260809 16:39:03.143185 3141 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:03.143224 3141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:03.143356 3141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:03.144342 3141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data/instance:
uuid: "ab9584656c28422197273579e70e0a0b"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "95dc3d7739ee83a7f94ffa32aa43c0bd"
server_key_iv: "040e33783776c2638754acc625056751"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.144627 3141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal/instance:
uuid: "ab9584656c28422197273579e70e0a0b"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "95dc3d7739ee83a7f94ffa32aa43c0bd"
server_key_iv: "040e33783776c2638754acc625056751"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.145773 3141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:39:03.146385 3164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:03.146528 3141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:39:03.146621 3141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "ab9584656c28422197273579e70e0a0b"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "95dc3d7739ee83a7f94ffa32aa43c0bd"
server_key_iv: "040e33783776c2638754acc625056751"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.146677 3141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:03.151140 3131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } attempt: 2
I20260809 16:39:03.151502 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.151390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60416)
0809 16:39:03.151438 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.151468 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 16:39:03.151815 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.151730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60428)
0809 16:39:03.151766 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.151791 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 16:39:03.151890 3131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:43079: Network error: Client connection negotiation failed: client connection to 127.31.15.253:43079: connect: Connection refused (error 111)
I20260809 16:39:03.162158 3141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:03.163969 3141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:03.164103 3141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:03.167992 3141 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.253:43079
I20260809 16:39:03.168031 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.253:43079 every 8 connection(s)
I20260809 16:39:03.168361 3141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
I20260809 16:39:03.169375 3244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:03.170902 3244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.173276 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3141
I20260809 16:39:03.173364 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-1/wal/instance
I20260809 16:39:03.173561 31807 external_mini_cluster.cc:1428] Setting key bff6175d13c4a98dd365d0188069ea97
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:39:03.180068 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.171599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.31.15.254:34569)
0809 16:39:03.171674 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.171676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.171859 (+ 183us) server_negotiation.cc:407] Connection header received
0809 16:39:03.172319 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.172320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.172333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.172373 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.172957 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.173486 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.174798 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.174995 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.175044 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.177242 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.177244 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.177244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.177252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.178740 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.179120 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.179121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.179121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.179187 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.179433 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.179434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.179434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.179581 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.179655 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.179875 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.179961 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 16:39:03.180131 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.171555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34569 (local address 127.0.0.1:48268)
0809 16:39:03.171744 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.171781 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.172228 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.172429 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.172432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.172695 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.172894 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.172898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.173572 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.173574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.174719 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.174723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.175078 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.175080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.175230 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.175753 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.175761 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.177084 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.178821 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.178828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.178830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.179049 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.179251 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.179251 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.179251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.179373 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.179643 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.179647 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.179759 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.179836 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.179954 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260809 16:39:03.180692 3244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.31.15.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.252@KRBTEST.COM" created.
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:03.187965 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.180957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60434)
0809 16:39:03.181013 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.181025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.181068 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.181297 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.181298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.181436 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.181492 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.181494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.182324 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.182325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.182828 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.182829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.183393 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.183393 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.183450 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.183745 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.183748 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.185406 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.186924 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.186925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.186926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.187139 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.187371 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.187372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.187372 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.187437 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.187741 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.187742 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.187777 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.187822 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.187900 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 16:39:03.188087 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.181020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60434 (local address 127.31.15.253:43079)
0809 16:39:03.181164 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.181169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.181189 (+ 20us) server_negotiation.cc:407] Connection header received
0809 16:39:03.181208 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.181209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.181220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.181278 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.181559 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.182241 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.182913 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.183327 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.183383 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.185525 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.185533 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.185533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.185539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.186848 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.187220 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.187223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.187223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.187316 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.187504 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.187505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.187505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.187679 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.187761 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.187857 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.187964 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260809 16:39:03.188741 3244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.189069 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.188959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60930)
0809 16:39:03.189005 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.189051 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:39:03.189217 3168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
I20260809 16:39:03.189632 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.189567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60944)
0809 16:39:03.189594 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.189617 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 16:39:03.189728 3244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42025: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
I20260809 16:39:03.199438 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:42025
--webserver_interface=127.31.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:44811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:39:03.238983 3244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } attempt: 1
I20260809 16:39:03.239559 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.239401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60954)
0809 16:39:03.239470 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.239521 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 16:39:03.239893 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.239810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60966)
0809 16:39:03.239845 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.239873 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:39:03.240000 3244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42025: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
I20260809 16:39:03.243040 3131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } attempt: 3
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:03.249858 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.243450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.31.15.253:43079)
0809 16:39:03.243510 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.243512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.243540 (+ 28us) server_negotiation.cc:407] Connection header received
0809 16:39:03.243567 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.243568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.243582 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.243632 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.243971 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.244473 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.245172 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.245411 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.245472 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.247432 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.247434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.247434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.247442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.248727 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.249099 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.249100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.249101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.249173 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.249356 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.249357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.249357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.249472 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.249540 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.249685 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.249761 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 16:39:03.249862 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.243387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60446)
0809 16:39:03.243437 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.243449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.243503 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.243644 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.243645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.243818 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.243893 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.243895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.244554 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.244555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.245089 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.245091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.245490 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.245491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.245569 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.246027 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.246032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.247339 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.248851 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.248852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.248853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.249028 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.249234 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.249235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.249235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.249299 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.249534 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.249535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.249585 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.249654 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.249756 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:39:03.250252 3131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.250622 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.250516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60982)
0809 16:39:03.250566 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.250600 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:39:03.250905 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.250844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:60992)
0809 16:39:03.250867 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.250891 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260809 16:39:03.251008 3131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42025: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
W20260809 16:39:03.282027 3256 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:03.282219 3256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:03.282236 3256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:03.282254 3256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 16:39:03.282177 3131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } attempt: 1
I20260809 16:39:03.282634 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.282506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32776)
0809 16:39:03.282565 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.282600 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:39:03.283023 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.282931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32788)
0809 16:39:03.282973 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.283002 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:39:03.283115 3131 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42025: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
W20260809 16:39:03.283556 3256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:03.283597 3256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:03.283617 3256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:03.283635 3256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:03.283658 3256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:03.285130 3256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:44811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:34569,127.31.15.253:43079,127.31.15.252:42025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:42025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:03.285367 3256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:03.285583 3256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:03.290078 3262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:03.290079 3266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:03.290109 3261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:03.290159 3256 server_base.cc:1034] running on GCE node
I20260809 16:39:03.290499 3256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:03.290778 3256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:03.291922 3256 hybrid_clock.cc:648] HybridClock initialized: now 1786293543291904 us; error 33 us; skew 500 ppm
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:03.294294 3256 init.cc:377] Logged in from keytab as kudu/127.31.15.252@KRBTEST.COM (short username kudu)
I20260809 16:39:03.294731 3256 webserver.cc:460] Webserver started at http://127.31.15.252:44227/ using document root <none> and password file <none>
I20260809 16:39:03.294946 3256 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:03.294992 3256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:03.295115 3256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:03.296170 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data/instance:
uuid: "2c1df57e8a724454a4c5b70c2f4a84ac"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "857d76aea49763b58f0c60f29a6a2c90"
server_key_iv: "e29b8d4ddea63a954ce756b19cf33c84"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.296500 3256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal/instance:
uuid: "2c1df57e8a724454a4c5b70c2f4a84ac"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "857d76aea49763b58f0c60f29a6a2c90"
server_key_iv: "e29b8d4ddea63a954ce756b19cf33c84"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.297709 3256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:39:03.298336 3286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:03.298494 3256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:39:03.298579 3256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "2c1df57e8a724454a4c5b70c2f4a84ac"
format_stamp: "Formatted at 2026-08-09 16:39:03 on dist-test-slave-21b9"
server_key: "857d76aea49763b58f0c60f29a6a2c90"
server_key_iv: "e29b8d4ddea63a954ce756b19cf33c84"
server_key_version: "encryptionkey@0"
I20260809 16:39:03.298643 3256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:03.308169 3244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } attempt: 2
I20260809 16:39:03.308632 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.308493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32798)
0809 16:39:03.308554 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.308594 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 16:39:03.308969 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.308894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32808)
0809 16:39:03.308932 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.308952 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 16:39:03.309055 3244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42025: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42025: connect: Connection refused (error 111)
I20260809 16:39:03.313596 3256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:03.316253 3256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:03.316375 3256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:03.320330 3256 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.252:42025
I20260809 16:39:03.320362 3368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.252:42025 every 8 connection(s)
I20260809 16:39:03.320685 3256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
I20260809 16:39:03.321660 3371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:03.323137 3371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.324208 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3256
I20260809 16:39:03.324298 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786293272397626-31807-0/minicluster-data/master-2/wal/instance
I20260809 16:39:03.324474 31807 external_mini_cluster.cc:1428] Setting key af575c848ebd499fa5264ad8b04006ba
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:03.332024 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.323835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.31.15.254:34569)
0809 16:39:03.323885 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.323887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.324093 (+ 206us) server_negotiation.cc:407] Connection header received
0809 16:39:03.324670 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.324672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.324685 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.324731 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.325369 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.325839 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.326688 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.326879 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.326943 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.329254 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.329256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.329256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.329265 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.330538 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.330968 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.330969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.330969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.331034 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.331365 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.331366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.331367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.331536 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.331610 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.331856 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.331935 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 16:39:03.332141 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.323801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:34569 (local address 127.0.0.1:48272)
0809 16:39:03.323969 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.324005 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.324563 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.324797 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.324800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.325102 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.325281 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.325285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.325905 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.325906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.326614 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.326616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.326966 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.326968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.327092 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.327536 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.327544 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.329122 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.330634 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.330641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.330643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.330884 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.331128 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.331129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.331130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.331265 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.331608 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.331615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.331731 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.331811 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.331942 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260809 16:39:03.332670 3371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.333438 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.324840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.31.15.252:42025)
0809 16:39:03.325012 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.325016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.325041 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:39:03.325095 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.325098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.325122 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.325188 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.325767 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.326533 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.327232 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.327559 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.327617 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.330287 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.330295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.330295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.330302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.331963 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.332403 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.332406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.332406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.332462 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.332675 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.332675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.332676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.332857 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.332986 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.333277 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.333343 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:39:03.338894 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.332915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:43079 (local address 127.0.0.1:60460)
0809 16:39:03.332984 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.332994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.333033 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.333137 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.333138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.333264 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.333324 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.333326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.333991 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.333992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.334700 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.334703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.335108 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.335109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.335188 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.335526 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.335530 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.336669 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.338003 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.338004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.338004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.338198 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.338374 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.338374 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.338374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.338425 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.338674 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.338675 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.338719 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.338772 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.338838 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 16:39:03.338964 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.332994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.31.15.253:43079)
0809 16:39:03.333045 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.333047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.333054 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:39:03.333079 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.333080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.333090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.333130 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.333391 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.333925 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.334776 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.335037 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.335091 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.336730 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.336731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.336732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.336739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.337929 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.338269 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.338270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.338271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.338327 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.338481 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.338482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.338482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.338608 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.338671 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.338813 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.338878 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:39:03.339511 3371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:03.340765 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.334141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.31.15.254:34569)
0809 16:39:03.334191 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.334193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.334229 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:39:03.334293 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.334294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.334301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.334340 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.334823 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.335397 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.336028 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.336313 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.336382 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.338298 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.338299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.338300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.338306 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.339725 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.340081 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.340082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.340082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.340143 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.340294 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.340294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.340295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.340416 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.340580 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.340614 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.340683 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:03.345579 3374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.339718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32834)
0809 16:39:03.339755 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.339764 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.339799 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.339893 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.339894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.340030 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.340061 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.340063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.340811 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.340811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.341498 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.341500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.341793 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.341794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.341828 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.342177 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.342180 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.343308 (+ 1128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.344742 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.344743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.344743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.344891 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.345050 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.345051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.345051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.345101 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.345292 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.345293 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.345328 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.345387 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.345480 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 16:39:03.345628 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.339779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.31.15.252:42025)
0809 16:39:03.339811 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.339813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.339819 (+ 6us) server_negotiation.cc:407] Connection header received
0809 16:39:03.339839 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.339840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.339848 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.339889 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.340120 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.340747 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.341556 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.341743 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.341788 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.343372 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.343373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.343373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.343379 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.344672 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.344952 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.344953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.344953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.344999 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.345148 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.345149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.345149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.345243 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.345294 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.345412 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.345522 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":17}
I20260809 16:39:03.346848 3371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: Bootstrap starting.
I20260809 16:39:03.347759 3371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:03.348055 3371 log.cc:824] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:03.348285 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.341365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60464 (local address 127.31.15.253:43079)
0809 16:39:03.341405 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.341407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.341443 (+ 36us) server_negotiation.cc:407] Connection header received
0809 16:39:03.341486 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.341487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.341495 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.341554 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.342326 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.342845 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.343650 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.343837 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.343892 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.345986 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.345987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.345988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.345994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.347249 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.347608 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.347608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.347609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.347656 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.347843 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.347844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.347844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.347962 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.348066 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.348157 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.348225 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:39:03.348827 3371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: No bootstrap required, opened a new log
I20260809 16:39:03.349625 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:39:03.350380 3371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.350512 3371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1df57e8a724454a4c5b70c2f4a84ac, State: Initialized, Role: FOLLOWER
I20260809 16:39:03.350600 3371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.350895 3371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:03.350860 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.350950 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.350696 3086 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "877a763676df45988bea8851f8d6fe39"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48288
W20260809 16:39:03.352581 3401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:03.352634 3401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:39:03.352880 3256 master_runner.cc:186] Error getting master registration for 127.31.15.254:34569: OK, instance_id { permanent_uuid: "877a763676df45988bea8851f8d6fe39" instance_seqno: 1786293543018159 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:39:03.355641 3256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:39:03.358258 3131 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } attempt: 2
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293542, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:03.365178 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.358524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32850)
0809 16:39:03.358572 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.358582 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.358634 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.358769 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.358770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.358940 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.358989 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.358990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.359821 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.359822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.360579 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.360582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.361025 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.361025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.361079 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.361469 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.361474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.362642 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.364080 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.364082 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.364082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.364297 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.364495 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.364496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.364496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.364566 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.364869 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.364870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.364915 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.364985 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.365076 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 16:39:03.365185 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.358583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.31.15.252:42025)
0809 16:39:03.358632 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.358633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.358642 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:39:03.358684 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.358685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.358695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.358751 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.359050 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.359748 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.360657 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.360948 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.361026 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.362722 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.362723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.362723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.362729 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.364015 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.364360 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.364361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.364361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.364405 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.364635 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.364637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.364637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.364805 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.364875 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.365023 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.365096 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:39:03.366724 3131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: Bootstrap starting.
I20260809 16:39:03.367467 3131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:03.367784 3131 log.cc:824] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:03.368723 3131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: No bootstrap required, opened a new log
I20260809 16:39:03.370203 3131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.370345 3131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877a763676df45988bea8851f8d6fe39, State: Initialized, Role: FOLLOWER
I20260809 16:39:03.370430 3131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.370769 3131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:03.370734 3409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.370842 3409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.370878 3086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:39:03.370973 3086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:03.371743 3086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.372238 3321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "877a763676df45988bea8851f8d6fe39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac"
I20260809 16:39:03.372376 3321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:39:03.372937 3425 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:03.373054 3425 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:39:03.373296 3321 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 877a763676df45988bea8851f8d6fe39 in term 1.
I20260809 16:39:03.373562 3042 leader_election.cc:302] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c1df57e8a724454a4c5b70c2f4a84ac, 877a763676df45988bea8851f8d6fe39; no voters:
I20260809 16:39:03.373673 3086 leader_election.cc:288] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [CANDIDATE]: Term 1 election: Requested vote from peers ab9584656c28422197273579e70e0a0b (127.31.15.253:43079), 2c1df57e8a724454a4c5b70c2f4a84ac (127.31.15.252:42025)
I20260809 16:39:03.373741 3409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:39:03.373893 3409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [term 1 LEADER]: Becoming Leader. State: Replica: 877a763676df45988bea8851f8d6fe39, State: Running, Role: LEADER
I20260809 16:39:03.373986 3409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.374372 3409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877a763676df45988bea8851f8d6fe39. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.374441 3409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:03.375996 3011 master_runner.cc:186] Error getting master registration for 127.31.15.253:43079: OK, instance_id { permanent_uuid: "ab9584656c28422197273579e70e0a0b" instance_seqno: 1786293543166909 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:39:03.377545 3011 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:39:03.377625 3437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:39:03.377842 3437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:39:03.388371 3321 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [term 1 FOLLOWER]: Refusing update from remote peer 877a763676df45988bea8851f8d6fe39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:39:03.388664 3409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:39:03.389256 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.374785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.31.15.252:42025)
0809 16:39:03.374819 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.374821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.375988 (+ 1167us) server_negotiation.cc:407] Connection header received
0809 16:39:03.378594 (+ 2606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.378595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.378606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.378646 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.380552 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.381219 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.382228 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.382501 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.382557 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.385653 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.385654 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.385654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.385662 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.387166 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.387919 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.387933 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.387933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.388001 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.388297 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.388298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.388299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.388441 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.388530 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.389070 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.389170 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 16:39:03.390261 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877a763676df45988bea8851f8d6fe39. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.390345 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.391301 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.375244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48298 (local address 127.31.15.254:34569)
0809 16:39:03.375302 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.375304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.379384 (+ 4080us) server_negotiation.cc:407] Connection header received
0809 16:39:03.379469 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.379470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.379480 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.379515 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.380104 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.380797 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.383072 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.383379 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.383435 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.384842 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.384843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.384843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.384850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.386702 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.390111 (+ 3409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.390112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.390113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.390173 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.390577 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.390578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.390579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.390718 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.390809 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.391124 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.391196 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 16:39:03.395635 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.375467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.31.15.253:43079)
0809 16:39:03.375576 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.375578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.379429 (+ 3851us) server_negotiation.cc:407] Connection header received
0809 16:39:03.379487 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.379488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.379511 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.379558 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.388106 (+ 8548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.388770 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.390589 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.390879 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.390919 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.392883 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.392884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.392885 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.392891 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.394359 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.394774 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.394775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.394775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.394847 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.395086 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.395087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.395087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.395208 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.395305 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.395476 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.395549 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 16:39:03.400193 3244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } attempt: 3
I20260809 16:39:03.400291 3409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.400377 3409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:03.400452 3443 mvcc.cc:205] Tried to move back new op lower bound from 7316658353715900416 to 7316658353660526592. Current Snapshot: MvccSnapshot[applied={T|T < 7316658353715900416}]
I20260809 16:39:03.400842 3379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.400913 3379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.401659 3445 mvcc.cc:205] Tried to move back new op lower bound from 7316658353715900416 to 7316658353660526592. Current Snapshot: MvccSnapshot[applied={T|T < 7316658353715900416}]
I20260809 16:39:03.403021 3437 catalog_manager.cc:1347] Generated new cluster ID: e14cdee429e94ac9a6e4e31b7543bc06
I20260809 16:39:03.403069 3437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 16:39:03 dist-test-slave-21b9 krb5kdc[2992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293543, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:03.409051 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.401231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42025 (local address 127.0.0.1:32866)
0809 16:39:03.401301 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:03.401311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:03.401352 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:03.401496 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:03.401497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:03.401669 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:03.401710 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.401712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.402609 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.402610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.403350 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:03.403353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.403816 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.403816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:03.403867 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.404268 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:03.404273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:03.405602 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:03.407964 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.407965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.407966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.408176 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.408423 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:03.408424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:03.408424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:03.408488 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:03.408760 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:03.408761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:03.408813 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 16:39:03.408865 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 16:39:03.408949 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 16:39:03.409401 3376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:03.401265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.31.15.252:42025)
0809 16:39:03.401310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.401312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.401358 (+ 46us) server_negotiation.cc:407] Connection header received
0809 16:39:03.401397 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.401397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.401417 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.401459 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.401847 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.402509 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.403440 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:03.403727 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:03.403774 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:03.405696 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:03.405697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:03.405698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:03.405704 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:03.407876 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.408254 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.408254 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.408255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.408346 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:03.408551 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:03.408552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:03.408552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:03.408677 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:03.408728 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:03.409254 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 16:39:03.409324 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 16:39:03.410758 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: Bootstrap starting.
I20260809 16:39:03.411501 3244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:03.411787 3244 log.cc:824] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:03.423949 3244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: No bootstrap required, opened a new log
I20260809 16:39:03.426174 3244 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.426321 3244 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9584656c28422197273579e70e0a0b, State: Initialized, Role: FOLLOWER
I20260809 16:39:03.426409 3244 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } }
I20260809 16:39:03.426977 3199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:03.427202 3437 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:39:03.427942 3437 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:39:03.427978 3199 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [term 1 FOLLOWER]: Refusing update from remote peer 877a763676df45988bea8851f8d6fe39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:39:03.428252 3409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:39:03.427016 3200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "877a763676df45988bea8851f8d6fe39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ab9584656c28422197273579e70e0a0b"
I20260809 16:39:03.426890 3244 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: configured and running, proceeding with master startup.
W20260809 16:39:03.431320 3480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:03.431375 3480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:39:03.433059 3456 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.433142 3456 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.433192 3456 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877a763676df45988bea8851f8d6fe39. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.433236 3456 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.433260 3456 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.433295 3456 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:03.433377 3437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39: Generated new TSK 0
I20260809 16:39:03.433570 3437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 16:39:03.440086 3427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "877a763676df45988bea8851f8d6fe39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "877a763676df45988bea8851f8d6fe39" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 34569 } } peers { permanent_uuid: "ab9584656c28422197273579e70e0a0b" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 43079 } } peers { permanent_uuid: "2c1df57e8a724454a4c5b70c2f4a84ac" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42025 } } }
I20260809 16:39:03.440177 3427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877a763676df45988bea8851f8d6fe39 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:03.440321 3200 raft_consensus.cc:2396] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 877a763676df45988bea8851f8d6fe39 for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 16:39:03.441852 3461 mvcc.cc:205] Tried to move back new op lower bound from 7316658353871204352 to 7316658353660526592. Current Snapshot: MvccSnapshot[applied={T|T < 7316658353715900416}]
W20260809 16:39:03.461890 3135 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.460539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.31.15.254:34569)
0809 16:39:03.460584 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.460586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.461221 (+ 635us) server_negotiation.cc:407] Connection header received
0809 16:39:03.461326 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.461328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.461340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.461370 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.461816 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48312: BlockingRecv error: recv got EOF from 127.0.0.1:48312 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:39:03.462805 3376 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.460928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32874 (local address 127.31.15.252:42025)
0809 16:39:03.460965 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.460966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.462287 (+ 1321us) server_negotiation.cc:407] Connection header received
0809 16:39:03.462382 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.462383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.462393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.462422 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.462743 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32874: BlockingRecv error: recv got EOF from 127.0.0.1:32874 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:39:03.463981 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.463362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.31.15.253:43079)
0809 16:39:03.463405 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.463407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.463505 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:39:03.463594 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.463595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.463605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.463631 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.463919 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60488: BlockingRecv error: recv got EOF from 127.0.0.1:60488 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:39:03.466316 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.465637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.31.15.253:43079)
0809 16:39:03.465668 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.465670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.465834 (+ 164us) server_negotiation.cc:407] Connection header received
0809 16:39:03.465934 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.465935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.465951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.465978 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.466274 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60498: BlockingRecv error: recv got EOF from 127.0.0.1:60498 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:39:03.469300 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.468588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60500 (local address 127.31.15.253:43079)
0809 16:39:03.468618 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.468620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.468823 (+ 203us) server_negotiation.cc:407] Connection header received
0809 16:39:03.468917 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.468918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.468928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.468954 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.469257 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60500: BlockingRecv error: recv got EOF from 127.0.0.1:60500 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:39:03.473104 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.472395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.31.15.253:43079)
0809 16:39:03.472426 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.472428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.472658 (+ 230us) server_negotiation.cc:407] Connection header received
0809 16:39:03.472744 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.472745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.472754 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.472780 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.473062 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60502: BlockingRecv error: recv got EOF from 127.0.0.1:60502 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 16:39:03.478879 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.478209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60504 (local address 127.31.15.253:43079)
0809 16:39:03.478239 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.478240 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.478421 (+ 181us) server_negotiation.cc:407] Connection header received
0809 16:39:03.478520 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.478520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.478530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.478557 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.478840 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60504: BlockingRecv error: recv got EOF from 127.0.0.1:60504 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:39:03.489045 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.488300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.31.15.253:43079)
0809 16:39:03.488333 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.488335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.488557 (+ 222us) server_negotiation.cc:407] Connection header received
0809 16:39:03.488647 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.488648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.488658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.488685 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.488996 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60506: BlockingRecv error: recv got EOF from 127.0.0.1:60506 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 16:39:03.507547 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.506467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.31.15.253:43079)
0809 16:39:03.506509 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.506512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.506773 (+ 261us) server_negotiation.cc:407] Connection header received
0809 16:39:03.506878 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.506879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.506891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.506936 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.507477 (+ 541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60510: BlockingRecv error: recv got EOF from 127.0.0.1:60510 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:39:03.541972 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.541159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.31.15.253:43079)
0809 16:39:03.541204 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.541206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.541393 (+ 187us) server_negotiation.cc:407] Connection header received
0809 16:39:03.541435 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.541436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.541450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.541521 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.541909 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60526: BlockingRecv error: recv got EOF from 127.0.0.1:60526 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 16:39:03.607739 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.607041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60530 (local address 127.31.15.253:43079)
0809 16:39:03.607107 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.607109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.607190 (+ 81us) server_negotiation.cc:407] Connection header received
0809 16:39:03.607244 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.607245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.607256 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.607362 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.607675 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60530: BlockingRecv error: recv got EOF from 127.0.0.1:60530 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 16:39:03.737540 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.736898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.31.15.253:43079)
0809 16:39:03.736965 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.736968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.737057 (+ 89us) server_negotiation.cc:407] Connection header received
0809 16:39:03.737109 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.737110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.737120 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.737172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.737462 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60546: BlockingRecv error: recv got EOF from 127.0.0.1:60546 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 16:39:03.995424 3250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:03.994804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.31.15.253:43079)
0809 16:39:03.994875 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:03.994877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:03.994888 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:39:03.994922 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:03.994923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:03.994936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:03.994982 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:03.995353 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60556: BlockingRecv error: recv got EOF from 127.0.0.1:60556 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 16:39:04.353015 3401 catalog_manager.cc:1259] Loaded cluster ID: e14cdee429e94ac9a6e4e31b7543bc06
I20260809 16:39:04.353094 3401 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: loading cluster ID for follower catalog manager: success
I20260809 16:39:04.353907 3401 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: acquiring CA information for follower catalog manager: success
I20260809 16:39:04.354211 3401 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c1df57e8a724454a4c5b70c2f4a84ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:39:04.431840 3480 catalog_manager.cc:1259] Loaded cluster ID: e14cdee429e94ac9a6e4e31b7543bc06
I20260809 16:39:04.431921 3480 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: loading cluster ID for follower catalog manager: success
I20260809 16:39:04.432721 3480 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: acquiring CA information for follower catalog manager: success
I20260809 16:39:04.433058 3480 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ab9584656c28422197273579e70e0a0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:39:04.510854 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:04.508551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.31.15.253:43079)
0809 16:39:04.508678 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:04.508680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:04.508712 (+ 32us) server_negotiation.cc:407] Connection header received
0809 16:39:04.508778 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:04.508779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:04.508832 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:04.508902 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:04.509025 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.509524 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.510230 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.510413 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.510455 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:04.510571 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:04.510682 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:04.510736 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:04.510752 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 16:39:04.510768 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 16:39:04.514855 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:04.512593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.31.15.252:42025)
0809 16:39:04.512702 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:04.512705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:04.512730 (+ 25us) server_negotiation.cc:407] Connection header received
0809 16:39:04.512775 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:04.512775 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:04.512820 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:04.512882 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:04.512986 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.513454 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.514143 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.514355 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.514411 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:04.514549 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:04.514648 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:04.514710 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:04.514751 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 16:39:04.514772 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":46,"threads_started":1}
I20260809 16:39:04.518971 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:04.516292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.31.15.253:43079)
0809 16:39:04.516333 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:04.516389 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:04.516589 (+ 200us) server_negotiation.cc:407] Connection header received
0809 16:39:04.516661 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:04.516662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:04.516672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:04.516718 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:04.516812 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.517553 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.518379 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.518562 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.518616 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:04.518747 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:04.518780 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:04.518829 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:04.518881 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:39:04.518902 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 16:39:04.519531 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:04.516368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.31.15.252:42025)
0809 16:39:04.516422 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:04.516434 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:04.516638 (+ 204us) server_negotiation.cc:407] Connection header received
0809 16:39:04.516684 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:04.516685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:04.516693 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:04.516739 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:04.516854 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.517551 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.518775 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.519047 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.519105 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:04.519295 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:04.519326 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:04.519367 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:04.519433 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 16:39:04.519453 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 16:39:04.519598 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:04.516276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.31.15.254:34569)
0809 16:39:04.516519 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:04.516521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:04.516750 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:39:04.516868 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:04.516869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:04.516916 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:04.516960 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:04.517126 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.517616 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.518697 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:04.518964 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:04.519022 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:04.519246 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:04.519371 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:04.519402 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:04.519490 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:39:04.519514 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":49,"threads_started":1}
I20260809 16:39:04.521245 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3011
I20260809 16:39:04.531332 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3141
I20260809 16:39:04.542671 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3256
2026-08-09T16:39:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2902 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 16:39:04 dist-test-slave-21b9 krb5kdc[3618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 16:39:04 dist-test-slave-21b9 krb5kdc[3618](info): set up 2 sockets
Aug 09 16:39:04 dist-test-slave-21b9 krb5kdc[3618](info): commencing operation
krb5kdc: starting...
W20260809 16:39:05.700248 31807 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/test-user.keytab.
Aug 09 16:39:05 dist-test-slave-21b9 krb5kdc[3618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T16:39:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T16:39:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.31.15.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
Entry for principal kudu/127.31.15.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab.
I20260809 16:39:05.817412 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46343
--webserver_interface=127.31.15.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 16:39:05.897990 3643 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:05.898213 3643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:05.898239 3643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:05.898264 3643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:05.899415 3643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:05.899456 3643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:05.899474 3643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:05.899489 3643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:05.899506 3643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:05.901001 3643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.254.keytab
--principal=kudu/127.31.15.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.254:46343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.31.15.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:05.901244 3643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:05.901456 3643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:05.906137 3658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:05.906286 3643 server_base.cc:1034] running on GCE node
W20260809 16:39:05.906123 3654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:05.906116 3655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:05.906677 3643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:05.906953 3643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:05.908089 3643 hybrid_clock.cc:648] HybridClock initialized: now 1786293545908080 us; error 29 us; skew 500 ppm
Aug 09 16:39:05 dist-test-slave-21b9 krb5kdc[3618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:05.914350 3643 init.cc:377] Logged in from keytab as kudu/127.31.15.254@KRBTEST.COM (short username kudu)
I20260809 16:39:05.914778 3643 webserver.cc:460] Webserver started at http://127.31.15.254:34481/ using document root <none> and password file <none>
I20260809 16:39:05.914975 3643 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:05.915019 3643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:05.915117 3643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:05.916255 3643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data/instance:
uuid: "0eee1270e7214ac898d59de4e548faf2"
format_stamp: "Formatted at 2026-08-09 16:39:05 on dist-test-slave-21b9"
server_key: "d303cc465e3fe3e6d901f0124ca110c1"
server_key_iv: "864103619de545e221b84cc5ce74357d"
server_key_version: "encryptionkey@0"
I20260809 16:39:05.916584 3643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance:
uuid: "0eee1270e7214ac898d59de4e548faf2"
format_stamp: "Formatted at 2026-08-09 16:39:05 on dist-test-slave-21b9"
server_key: "d303cc465e3fe3e6d901f0124ca110c1"
server_key_iv: "864103619de545e221b84cc5ce74357d"
server_key_version: "encryptionkey@0"
I20260809 16:39:05.917867 3643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 16:39:05.918524 3668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:05.918672 3643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:39:05.918756 3643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal
uuid: "0eee1270e7214ac898d59de4e548faf2"
format_stamp: "Formatted at 2026-08-09 16:39:05 on dist-test-slave-21b9"
server_key: "d303cc465e3fe3e6d901f0124ca110c1"
server_key_iv: "864103619de545e221b84cc5ce74357d"
server_key_version: "encryptionkey@0"
I20260809 16:39:05.918810 3643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:05.941012 3643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:05.942768 3643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:05.942876 3643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:05.946632 3643 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.254:46343
I20260809 16:39:05.946671 3742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.254:46343 every 8 connection(s)
I20260809 16:39:05.947010 3643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/data/info.pb
I20260809 16:39:05.947979 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:05.949504 3743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:05.952076 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3643
I20260809 16:39:05.952183 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-0/wal/instance
I20260809 16:39:05.952378 31807 external_mini_cluster.cc:1428] Setting key f929e66c7415c9ccf32bda38668b3aeb
Aug 09 16:39:05 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:39:05.958971 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:05.950120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46343 (local address 127.0.0.1:49794)
0809 16:39:05.950274 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:05.950308 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:05.950773 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:05.951029 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:05.951032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:05.951321 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:05.951482 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:05.951487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:05.952817 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:05.952818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:05.953488 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:05.953490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:05.954135 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:05.954137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:05.954215 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:05.954650 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:05.954661 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:05.956116 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:05.957641 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:05.957647 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:05.957649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:05.957829 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:05.958035 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:05.958036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:05.958037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:05.958114 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:05.958419 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:05.958424 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:05.958484 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 16:39:05.958567 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 16:39:05.958783 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":84,"thread_start_us":74,"threads_started":1}
I20260809 16:39:05.958976 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:05.950276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.31.15.254:46343)
0809 16:39:05.950376 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:05.950380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:05.950411 (+ 31us) server_negotiation.cc:407] Connection header received
0809 16:39:05.950914 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:05.950916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:05.950938 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:05.951022 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:05.951577 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:05.952710 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:05.953566 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:05.954019 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:05.954132 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:05.956193 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:05.956200 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:05.956201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:05.956206 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:05.957566 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:05.957897 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:05.957900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:05.957901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:05.957973 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:05.958167 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:05.958168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:05.958169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:05.958359 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:05.958429 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:05.958610 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 16:39:05.958711 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":40,"threads_started":1}
I20260809 16:39:05.959704 3743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:05.960033 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:05.959906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36896)
0809 16:39:05.959957 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:05.960006 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 16:39:05.960181 3671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.31.15.253@KRBTEST.COM; defaulting to no policy
I20260809 16:39:05.960578 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:05.960481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36906)
0809 16:39:05.960532 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:05.960560 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 16:39:05.960681 3743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:39029: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
Entry for principal kudu/127.31.15.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab.
I20260809 16:39:05.977734 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:39029
--webserver_interface=127.31.15.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:39:06.009903 3743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } attempt: 1
I20260809 16:39:06.010488 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.010303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36922)
0809 16:39:06.010355 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.010431 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 16:39:06.012125 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.011967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36934)
0809 16:39:06.012034 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.012092 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 16:39:06.012758 3743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:39029: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
W20260809 16:39:06.059684 3754 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:06.059893 3754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:06.059909 3754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:06.059929 3754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:06.061041 3754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:06.061084 3754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:06.061098 3754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:06.061121 3754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:06.061138 3754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:06.062631 3754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.253.keytab
--principal=kudu/127.31.15.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.253:39029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.31.15.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:06.062860 3754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:06.063071 3754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:06.067737 3761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:06.067777 3760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:06.067778 3754 server_base.cc:1034] running on GCE node
W20260809 16:39:06.067746 3766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:06.068186 3754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:06.068464 3754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:06.069605 3754 hybrid_clock.cc:648] HybridClock initialized: now 1786293546069592 us; error 28 us; skew 500 ppm
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:06.072108 3754 init.cc:377] Logged in from keytab as kudu/127.31.15.253@KRBTEST.COM (short username kudu)
I20260809 16:39:06.072535 3754 webserver.cc:460] Webserver started at http://127.31.15.253:37815/ using document root <none> and password file <none>
I20260809 16:39:06.072738 3754 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:06.072785 3754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:06.072908 3754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:06.073974 3754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data/instance:
uuid: "14b4171caef44fd98d9c85e878fdd61d"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "21e543800c843463b69c8e53cfeb7878"
server_key_iv: "8fc70fd6ca833cbc5176e492b48a87e6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.074313 3754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal/instance:
uuid: "14b4171caef44fd98d9c85e878fdd61d"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "21e543800c843463b69c8e53cfeb7878"
server_key_iv: "8fc70fd6ca833cbc5176e492b48a87e6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.075595 3754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 16:39:06.076246 3775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:06.076400 3754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 16:39:06.076483 3754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal
uuid: "14b4171caef44fd98d9c85e878fdd61d"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "21e543800c843463b69c8e53cfeb7878"
server_key_iv: "8fc70fd6ca833cbc5176e492b48a87e6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.076541 3754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:06.080920 3743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } attempt: 2
I20260809 16:39:06.081277 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.081163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36940)
0809 16:39:06.081215 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.081247 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 16:39:06.081596 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.081507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36956)
0809 16:39:06.081542 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.081566 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 16:39:06.081676 3743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.253:39029: Network error: Client connection negotiation failed: client connection to 127.31.15.253:39029: connect: Connection refused (error 111)
I20260809 16:39:06.124926 3754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:06.127111 3754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:06.127250 3754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:06.131091 3754 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.253:39029
I20260809 16:39:06.131134 3851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.253:39029 every 8 connection(s)
I20260809 16:39:06.131484 3754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/data/info.pb
I20260809 16:39:06.132452 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:06.132802 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3754
I20260809 16:39:06.132896 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-1/wal/instance
I20260809 16:39:06.133080 31807 external_mini_cluster.cc:1428] Setting key 0bcf69aa26ae1e499cb6a479e5c15252
I20260809 16:39:06.134081 3852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.31.15.252@KRBTEST.COM; defaulting to no policy
I20260809 16:39:06.143900 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.134775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.31.15.254:46343)
0809 16:39:06.134827 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.134829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.135075 (+ 246us) server_negotiation.cc:407] Connection header received
0809 16:39:06.135731 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.135732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.135742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.135786 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.136372 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.136865 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.138267 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.138477 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.138540 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.141035 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.141037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.141038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.141055 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.142542 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.142929 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.142929 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.142930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.142977 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.143221 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.143223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.143223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.143390 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.143453 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.143721 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.143806 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 16:39:06.144059 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.134745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46343 (local address 127.0.0.1:49798)
0809 16:39:06.134933 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.134985 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.135631 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.135859 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.135864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.136102 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.136297 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.136301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.136954 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.136955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.138185 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.138188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.138579 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.138580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.138740 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.139217 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.139227 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.140940 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.142631 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.142639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.142641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.142860 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.143036 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.143036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.143036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.143138 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.143451 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.143459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.143598 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.143688 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.143840 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.31.15.252@KRBTEST.COM" created.
I20260809 16:39:06.144723 3852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:06.152992 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.144979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36972)
0809 16:39:06.145026 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.145037 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.145082 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.145370 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.145370 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.145478 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.145541 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.145542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.146636 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.146637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.147393 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.147395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.147886 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.147888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.147963 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.148343 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.148347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.149670 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.151790 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.151791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.151791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.151989 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.152201 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.152202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.152202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.152294 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.152661 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.152662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.152711 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.152798 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.152887 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:39:06.153146 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.145041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36972 (local address 127.31.15.253:39029)
0809 16:39:06.145220 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.145225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.145252 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:39:06.145275 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.145277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.145289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.145361 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.145603 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.146535 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.147466 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.147798 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.147859 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.149774 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.149783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.149783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.149789 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.151706 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.152065 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.152068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.152069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.152118 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.152357 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.152358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.152358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.152564 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.152641 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.152868 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.152990 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260809 16:39:06.154018 3852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:06.154393 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.154269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45304)
0809 16:39:06.154319 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.154370 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 16:39:06.154593 3777 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
I20260809 16:39:06.155601 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.155522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45312)
0809 16:39:06.155551 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.155577 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 16:39:06.155696 3852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42397: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
Entry for principal kudu/127.31.15.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab.
I20260809 16:39:06.166618 31807 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskfvbJG8/build/release/bin/kudu
/tmp/dist-test-taskfvbJG8/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:42397
--webserver_interface=127.31.15.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.31.15.212:35209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 16:39:06.172817 3743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } attempt: 3
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:06.179802 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.173159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:36988)
0809 16:39:06.173211 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.173223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.173270 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.173389 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.173391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.173541 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.173605 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.173607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.174230 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.174231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.174798 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.174800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.175145 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.175146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.175225 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.175707 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.175712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.177091 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.178788 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.178789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.178790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.178974 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.179174 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.179174 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.179175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.179236 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.179535 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.179536 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.179582 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.179642 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.179724 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 16:39:06.179852 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.173224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.31.15.253:39029)
0809 16:39:06.173285 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.173287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.173296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:39:06.173321 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.173322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.173334 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.173379 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.173676 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.174157 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.174872 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.175085 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.175141 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.177190 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.177192 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.177192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.177200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.178682 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.179055 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.179056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.179056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.179114 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.179326 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.179327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.179328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.179458 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.179520 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.179675 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.179756 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:39:06.180183 3743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:06.180502 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.180406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45314)
0809 16:39:06.180448 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.180478 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 16:39:06.180750 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.180697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45328)
0809 16:39:06.180717 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.180738 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 16:39:06.180821 3743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42397: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
I20260809 16:39:06.204902 3852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } attempt: 1
I20260809 16:39:06.205428 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.205268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45342)
0809 16:39:06.205351 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.205395 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 16:39:06.205752 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.205675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45346)
0809 16:39:06.205710 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.205735 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 16:39:06.205854 3852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42397: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
I20260809 16:39:06.211966 3743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } attempt: 1
I20260809 16:39:06.212414 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.212282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45360)
0809 16:39:06.212342 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.212373 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 16:39:06.212786 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.212694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45362)
0809 16:39:06.212737 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.212765 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 16:39:06.212877 3743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42397: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
W20260809 16:39:06.250464 3860 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 16:39:06.250701 3860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 16:39:06.250720 3860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 16:39:06.250738 3860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 16:39:06.251921 3860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 16:39:06.251962 3860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 16:39:06.251976 3860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 16:39:06.251987 3860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 16:39:06.252002 3860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 16:39:06.253450 3860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.31.15.212:35209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.31.15.254:46343,127.31.15.253:39029,127.31.15.252:42397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/krb5kdc/kudu_127.31.15.252.keytab
--principal=kudu/127.31.15.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.31.15.252:42397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.31.15.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 14:17:19 UTC on ccycloud.quasar-aikwje.root.comops.site
build id 4576541
I20260809 16:39:06.253643 3860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 16:39:06.253823 3860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 16:39:06.258383 3872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:06.258410 3871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 16:39:06.258410 3877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 16:39:06.258649 3860 server_base.cc:1034] running on GCE node
I20260809 16:39:06.258884 3860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 16:39:06.259202 3860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 16:39:06.260358 3860 hybrid_clock.cc:648] HybridClock initialized: now 1786293546260347 us; error 26 us; skew 500 ppm
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 16:39:06.262764 3860 init.cc:377] Logged in from keytab as kudu/127.31.15.252@KRBTEST.COM (short username kudu)
I20260809 16:39:06.263195 3860 webserver.cc:460] Webserver started at http://127.31.15.252:42703/ using document root <none> and password file <none>
I20260809 16:39:06.263433 3860 fs_manager.cc:359] Metadata directory not provided
I20260809 16:39:06.263480 3860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 16:39:06.263586 3860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 16:39:06.264588 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data/instance:
uuid: "c57d7a7f448340d2b86de8b7cf0d8892"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "5432022ed6d3ae4a417ded7ed94a35e2"
server_key_iv: "cf2953c43c68957a7ca85844de022ac6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.264886 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal/instance:
uuid: "c57d7a7f448340d2b86de8b7cf0d8892"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "5432022ed6d3ae4a417ded7ed94a35e2"
server_key_iv: "cf2953c43c68957a7ca85844de022ac6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.266100 3860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 16:39:06.266719 3886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 16:39:06.266877 3860 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 16:39:06.266963 3860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data,/tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal
uuid: "c57d7a7f448340d2b86de8b7cf0d8892"
format_stamp: "Formatted at 2026-08-09 16:39:06 on dist-test-slave-21b9"
server_key: "5432022ed6d3ae4a417ded7ed94a35e2"
server_key_iv: "cf2953c43c68957a7ca85844de022ac6"
server_key_version: "encryptionkey@0"
I20260809 16:39:06.267033 3860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 16:39:06.273988 3852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } attempt: 2
I20260809 16:39:06.274369 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.274259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45378)
0809 16:39:06.274311 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.274341 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:39:06.274670 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.274589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45384)
0809 16:39:06.274625 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.274648 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 16:39:06.274744 3852 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.31.15.252:42397: Network error: Client connection negotiation failed: client connection to 127.31.15.252:42397: connect: Connection refused (error 111)
I20260809 16:39:06.277130 3860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 16:39:06.278916 3860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 16:39:06.279047 3860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 16:39:06.282963 3860 rpc_server.cc:307] RPC server started. Bound to: 127.31.15.252:42397
I20260809 16:39:06.283011 3965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.31.15.252:42397 every 8 connection(s)
I20260809 16:39:06.283378 3860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/data/info.pb
I20260809 16:39:06.284334 3966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 16:39:06.285851 3966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } has no permanent_uuid. Determining permanent_uuid...
I20260809 16:39:06.288036 3743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } attempt: 2
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
I20260809 16:39:06.291821 31807 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskfvbJG8/build/release/bin/kudu as pid 3860
I20260809 16:39:06.291927 31807 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskfvbJG8/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786293272397626-31807-0/minicluster-data/master-2/wal/instance
I20260809 16:39:06.292136 31807 external_mini_cluster.cc:1428] Setting key 7e182804fcf984606b57c754f3601fc8
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.254@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:06.295547 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.286558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.31.15.254:46343)
0809 16:39:06.286646 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.286649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.286829 (+ 180us) server_negotiation.cc:407] Connection header received
0809 16:39:06.287290 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.287292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.287304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.287351 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.287913 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.288496 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.289812 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.290029 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.290094 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.292361 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.292363 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.292363 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.292370 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.293948 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.294399 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.294400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.294400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.294454 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.294728 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.294729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.294729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.294922 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.295009 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.295330 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.295444 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 16:39:06.295673 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.286533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.254:46343 (local address 127.0.0.1:49804)
0809 16:39:06.286716 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.286753 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.287202 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.287418 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.287421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.287665 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.287847 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.287851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.288564 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.288565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.289730 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.289733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.290116 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.290117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.290217 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.290698 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.290706 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.292272 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.294030 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.294037 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.294039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.294325 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.294512 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.294513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.294513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.294656 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.294995 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.295001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.295153 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.295261 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.295436 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":105,"threads_started":1}
I20260809 16:39:06.296325 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.288286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45394)
0809 16:39:06.288336 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.288348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.288392 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.288650 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.288651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.288781 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.288815 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.288816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.289987 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.289988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.290965 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.290968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.291548 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.291549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.291610 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.292035 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.292040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.293390 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.295204 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.295205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.295205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.295456 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.295676 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.295677 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.295678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.295732 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.296028 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.296029 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.296075 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.296144 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.296233 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 16:39:06.296350 3972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.288348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.31.15.252:42397)
0809 16:39:06.288496 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.288500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.288521 (+ 21us) server_negotiation.cc:407] Connection header received
0809 16:39:06.288542 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.288544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.288563 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.288643 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.288876 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.289920 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.291051 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.291457 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.291511 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.293489 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.293498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.293499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.293505 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.295141 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.295542 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.295545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.295546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.295610 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.295796 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.295797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.295797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.295952 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.296045 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.296186 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.296253 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260809 16:39:06.296463 3966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:06.298355 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: Bootstrap starting.
I20260809 16:39:06.299124 3743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:06.299502 3743 log.cc:824] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: Log is configured to *not* fsync() on all Append() calls
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:06.301316 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: No bootstrap required, opened a new log
I20260809 16:39:06.301501 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.292488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45400 (local address 127.31.15.252:42397)
0809 16:39:06.292625 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.292627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.292871 (+ 244us) server_negotiation.cc:407] Connection header received
0809 16:39:06.292946 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.292947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.292959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.293032 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.293711 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.294285 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.295258 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.295546 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.295853 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.298381 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.298382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.298383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.298389 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.300425 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.300723 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.300724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.300724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.300786 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.301006 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.301007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.301007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.301112 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.301265 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.301310 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.301380 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260809 16:39:06.303247 3743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.303411 3743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eee1270e7214ac898d59de4e548faf2, State: Initialized, Role: FOLLOWER
I20260809 16:39:06.303479 3743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.303750 3977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.303802 3743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:06.303858 3977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.304433 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.296685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.253:39029 (local address 127.0.0.1:37004)
0809 16:39:06.296727 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.296736 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.296766 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.296883 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.296884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.297019 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.297054 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.297056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.297775 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.297776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.298497 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.298499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.298975 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.298976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.299034 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.299407 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.299412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.300677 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.303331 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.303332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.303332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.303511 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.303776 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.303776 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.303777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.303834 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.304153 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.304154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.304196 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.304254 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.304340 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 16:39:06.304450 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.296755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37004 (local address 127.31.15.253:39029)
0809 16:39:06.296796 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.296797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.296804 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:39:06.296827 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.296828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.296835 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.296882 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.297139 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.297683 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.298596 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.298864 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.298920 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.300756 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.300758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.300758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.300765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.302810 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.303633 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.303634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.303635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.303698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.303979 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.303980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.303980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.304085 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.304144 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.304286 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.304352 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 16:39:06.305184 3966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } has no permanent_uuid. Determining permanent_uuid...
W20260809 16:39:06.305665 3996 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:06.305760 3996 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.254@KRBTEST.COM
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.252@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:06.307250 3643 master_runner.cc:186] Error getting master registration for 127.31.15.253:39029: OK, instance_id { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" instance_seqno: 1786293546130063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:39:06.310119 3643 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:39:06.311056 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.302139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49812 (local address 127.31.15.254:46343)
0809 16:39:06.302253 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.302255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.302263 (+ 8us) server_negotiation.cc:407] Connection header received
0809 16:39:06.302281 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.302282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.302291 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.302347 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.302879 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.303543 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.304468 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.304755 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.304818 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.307913 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.307914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.307914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.307921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.309773 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.310153 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.310154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.310154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.310212 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.310603 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.310604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.310604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.310713 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.310840 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.310927 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.311002 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 16:39:06.312793 3971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.305610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45406)
0809 16:39:06.305743 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.305752 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.305790 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.306018 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.306019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.306156 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.306189 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.306191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.307000 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.307000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.307720 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.307723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.308257 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.308258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.308302 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.308607 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.308612 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.309964 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.311779 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.311779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.311780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.311981 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.312193 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.312194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.312195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.312259 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.312481 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.312481 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.312517 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.312579 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.312672 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 16:39:06.312840 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.305856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45406 (local address 127.31.15.252:42397)
0809 16:39:06.305898 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.305900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.305909 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:39:06.305926 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.305927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.305937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.305993 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.306253 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.306913 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.307866 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.308157 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.308216 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.310047 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.310048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.310048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.310054 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.311710 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.312065 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.312066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.312066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.312124 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.312324 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.312325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.312325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.312426 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.312480 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.312603 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.312720 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":21}
I20260809 16:39:06.314594 3966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: Bootstrap starting.
I20260809 16:39:06.315397 3966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:06.315682 3966 log.cc:824] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: Log is configured to *not* fsync() on all Append() calls
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.31.15.253@KRBTEST.COM
I20260809 16:39:06.316380 3966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: No bootstrap required, opened a new log
I20260809 16:39:06.318807 3966 raft_consensus.cc:348] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.318962 3966 raft_consensus.cc:729] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c57d7a7f448340d2b86de8b7cf0d8892, State: Initialized, Role: FOLLOWER
I20260809 16:39:06.319067 3966 consensus_queue.cc:260] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.319398 3966 sys_catalog.cc:564] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:06.319442 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.311565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.31.15.253:39029)
0809 16:39:06.311599 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.311600 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.311607 (+ 7us) server_negotiation.cc:407] Connection header received
0809 16:39:06.311647 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.311648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.311656 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.311702 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.312243 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.312931 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.313763 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.314055 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.314128 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.316619 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.316620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.316620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.316626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.318156 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.318534 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.318535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.318535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.318596 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.318781 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.318782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.318782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.318918 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.319081 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.319224 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.319359 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:39:06.319643 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.319721 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.320986 31807 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 16:39:06.321918 3705 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0eee1270e7214ac898d59de4e548faf2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49812
I20260809 16:39:06.322048 3705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 16:39:06.322101 3705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 0 FOLLOWER]: Advancing to term 1
W20260809 16:39:06.322916 4021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:06.322968 4021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:39:06.323066 3705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.323328 3705 leader_election.cc:288] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [CANDIDATE]: Term 1 election: Requested vote from peers 14b4171caef44fd98d9c85e878fdd61d (127.31.15.253:39029), c57d7a7f448340d2b86de8b7cf0d8892 (127.31.15.252:42397)
I20260809 16:39:06.323601 3928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0eee1270e7214ac898d59de4e548faf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c57d7a7f448340d2b86de8b7cf0d8892"
I20260809 16:39:06.323699 3928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:06.325011 3928 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eee1270e7214ac898d59de4e548faf2 in term 1.
I20260809 16:39:06.325209 3670 leader_election.cc:302] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eee1270e7214ac898d59de4e548faf2, c57d7a7f448340d2b86de8b7cf0d8892; no voters:
I20260809 16:39:06.322679 3860 master_runner.cc:186] Error getting master registration for 127.31.15.253:39029: OK, instance_id { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" instance_seqno: 1786293546130063 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 16:39:06.325356 3860 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 16:39:06.325877 3977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 16:39:06.326081 3977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [term 1 LEADER]: Becoming Leader. State: Replica: 0eee1270e7214ac898d59de4e548faf2, State: Running, Role: LEADER
I20260809 16:39:06.326184 3977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.338687 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eee1270e7214ac898d59de4e548faf2. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.338872 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:06.339114 4036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 16:39:06.339362 4036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 16:39:06.339553 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.331513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.31.15.254:46343)
0809 16:39:06.331565 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.331567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.331796 (+ 229us) server_negotiation.cc:407] Connection header received
0809 16:39:06.331894 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.331895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.331904 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.331931 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.332667 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.333408 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.334411 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.334704 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.334775 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.336239 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.336241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.336241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.336248 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.338056 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.338423 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.338424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.338424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.338494 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.338789 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.338790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.338791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.338931 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.339024 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.339384 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.339469 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 16:39:06.341214 3928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [term 1 FOLLOWER]: Refusing update from remote peer 0eee1270e7214ac898d59de4e548faf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:39:06.341564 4031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:39:06.343374 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eee1270e7214ac898d59de4e548faf2. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.343477 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.350366 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.331951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45416 (local address 127.31.15.252:42397)
0809 16:39:06.331989 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.331990 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.339977 (+ 7987us) server_negotiation.cc:407] Connection header received
0809 16:39:06.340130 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.340131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.340154 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.340183 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.341550 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.342231 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.344732 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.345051 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.345090 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.347100 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.347101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.347102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.347121 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.348753 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.349313 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.349314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.349315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.349374 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.349666 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.349667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.349668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.349805 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.349885 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.350196 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.350275 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 16:39:06.351779 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.331516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37030 (local address 127.31.15.253:39029)
0809 16:39:06.331601 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.331603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.339401 (+ 7798us) server_negotiation.cc:407] Connection header received
0809 16:39:06.339508 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.339509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.339520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.339547 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.340526 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.341181 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.342707 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.343035 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.343101 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.348694 (+ 5593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.348696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.348696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.348703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.350412 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.350769 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.350770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.350770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.350843 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.351033 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.351035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.351035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.351173 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.351285 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.351608 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.351687 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 16:39:06.352329 3977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.352418 3977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:06.352550 4037 mvcc.cc:205] Tried to move back new op lower bound from 7316658365810847744 to 7316658365752807424. Current Snapshot: MvccSnapshot[applied={T|T < 7316658365810847744}]
I20260809 16:39:06.352686 4002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.352769 4002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.352922 4040 mvcc.cc:205] Tried to move back new op lower bound from 7316658365810847744 to 7316658365752807424. Current Snapshot: MvccSnapshot[applied={T|T < 7316658365810847744}]
I20260809 16:39:06.363727 4036 catalog_manager.cc:1347] Generated new cluster ID: 43629c53828b4116963e1a90d9eadfb8
I20260809 16:39:06.363798 4036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 16:39:06.365823 3852 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } attempt: 3
Aug 09 16:39:06 dist-test-slave-21b9 krb5kdc[3618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786293546, etypes {rep=17 tkt=17 ses=17}, kudu/127.31.15.253@KRBTEST.COM for kudu/127.31.15.252@KRBTEST.COM
I20260809 16:39:06.383625 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.366132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.31.15.252:42397 (local address 127.0.0.1:45424)
0809 16:39:06.366216 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 16:39:06.366227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 16:39:06.366262 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 16:39:06.375507 (+ 9245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 16:39:06.375508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 16:39:06.375699 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 16:39:06.375753 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.375755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.376685 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.376686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.377574 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 16:39:06.377577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.378634 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.378635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 16:39:06.378694 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.379204 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 16:39:06.379209 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 16:39:06.380829 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 16:39:06.382550 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.382551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.382552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.382748 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.382945 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 16:39:06.382946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 16:39:06.382947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 16:39:06.383025 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 16:39:06.383313 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 16:39:06.383314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 16:39:06.383372 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 16:39:06.383427 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 16:39:06.383522 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 16:39:06.384166 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:06.375331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.31.15.252:42397)
0809 16:39:06.375382 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.375384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.375393 (+ 9us) server_negotiation.cc:407] Connection header received
0809 16:39:06.375414 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.375415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.375425 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.375486 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.375843 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.376590 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.378175 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:06.378459 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:06.378517 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:06.380917 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 16:39:06.380918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 16:39:06.380918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 16:39:06.380925 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 16:39:06.382464 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.382815 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.382816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.382816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.382866 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 16:39:06.383085 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 16:39:06.383085 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 16:39:06.383086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 16:39:06.383203 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 16:39:06.383255 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:06.384003 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 16:39:06.384080 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 16:39:06.386009 3852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: Bootstrap starting.
I20260809 16:39:06.386744 3852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: Neither blocks nor log segments found. Creating new log.
I20260809 16:39:06.387058 3852 log.cc:824] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: Log is configured to *not* fsync() on all Append() calls
I20260809 16:39:06.391754 3852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: No bootstrap required, opened a new log
I20260809 16:39:06.393805 3852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.393957 3852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14b4171caef44fd98d9c85e878fdd61d, State: Initialized, Role: FOLLOWER
I20260809 16:39:06.394042 3852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } }
I20260809 16:39:06.394373 3852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 16:39:06.394824 3815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0eee1270e7214ac898d59de4e548faf2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "14b4171caef44fd98d9c85e878fdd61d"
I20260809 16:39:06.394929 3815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [term 0 FOLLOWER]: Advancing to term 1
I20260809 16:39:06.395619 4036 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 16:39:06.395788 3815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eee1270e7214ac898d59de4e548faf2 in term 1.
I20260809 16:39:06.396005 3814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [term 1 FOLLOWER]: Refusing update from remote peer 0eee1270e7214ac898d59de4e548faf2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 16:39:06.396447 4036 catalog_manager.cc:1504] Loading token signing keys...
I20260809 16:39:06.396750 4031 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 16:39:06.398255 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.398940 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.399073 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0eee1270e7214ac898d59de4e548faf2. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.399191 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.399288 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.399392 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 16:39:06.400624 4067 mvcc.cc:205] Tried to move back new op lower bound from 7316658366017204224 to 7316658365752807424. Current Snapshot: MvccSnapshot[applied={T|T < 7316658365810847744}]
W20260809 16:39:06.402288 4071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 16:39:06.402388 4071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 16:39:06.404369 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0eee1270e7214ac898d59de4e548faf2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0eee1270e7214ac898d59de4e548faf2" member_type: VOTER last_known_addr { host: "127.31.15.254" port: 46343 } } peers { permanent_uuid: "14b4171caef44fd98d9c85e878fdd61d" member_type: VOTER last_known_addr { host: "127.31.15.253" port: 39029 } } peers { permanent_uuid: "c57d7a7f448340d2b86de8b7cf0d8892" member_type: VOTER last_known_addr { host: "127.31.15.252" port: 42397 } } }
I20260809 16:39:06.404461 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2 [sys.catalog]: This master's current role is: LEADER
I20260809 16:39:06.405694 4036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0eee1270e7214ac898d59de4e548faf2: Generated new TSK 0
I20260809 16:39:06.405866 4036 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 16:39:06.428531 3748 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.426745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.31.15.254:46343)
0809 16:39:06.426793 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.426795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.427837 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 16:39:06.427948 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.427950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.427962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.428006 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.428455 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49830: BlockingRecv error: recv got EOF from 127.0.0.1:49830 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 16:39:06.429508 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.427196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.31.15.253:39029)
0809 16:39:06.427236 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.427238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.428942 (+ 1704us) server_negotiation.cc:407] Connection header received
0809 16:39:06.429056 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.429057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.429068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.429099 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.429446 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37036: BlockingRecv error: recv got EOF from 127.0.0.1:37036 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 16:39:06.430528 3975 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.427566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45434 (local address 127.31.15.252:42397)
0809 16:39:06.427606 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.427608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.429909 (+ 2301us) server_negotiation.cc:407] Connection header received
0809 16:39:06.430036 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.430037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.430048 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.430076 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.430470 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45434: BlockingRecv error: recv got EOF from 127.0.0.1:45434 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 16:39:06.432962 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.432231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.31.15.253:39029)
0809 16:39:06.432276 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.432278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.432480 (+ 202us) server_negotiation.cc:407] Connection header received
0809 16:39:06.432582 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.432583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.432601 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.432628 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.432921 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37044: BlockingRecv error: recv got EOF from 127.0.0.1:37044 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:39:06.436002 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.435199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.31.15.253:39029)
0809 16:39:06.435229 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.435230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.435546 (+ 316us) server_negotiation.cc:407] Connection header received
0809 16:39:06.435634 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.435635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.435644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.435669 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.435947 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37060: BlockingRecv error: recv got EOF from 127.0.0.1:37060 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 16:39:06.439803 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.439064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.31.15.253:39029)
0809 16:39:06.439096 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.439097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.439353 (+ 256us) server_negotiation.cc:407] Connection header received
0809 16:39:06.439440 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.439441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.439450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.439476 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.439762 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37076: BlockingRecv error: recv got EOF from 127.0.0.1:37076 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 16:39:06.445600 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.444855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.31.15.253:39029)
0809 16:39:06.444886 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.444888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.445138 (+ 250us) server_negotiation.cc:407] Connection header received
0809 16:39:06.445229 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.445230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.445240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.445269 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.445561 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37090: BlockingRecv error: recv got EOF from 127.0.0.1:37090 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 16:39:06.455947 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.454814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.31.15.253:39029)
0809 16:39:06.454850 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.454851 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.455431 (+ 580us) server_negotiation.cc:407] Connection header received
0809 16:39:06.455532 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.455533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.455543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.455572 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.455902 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37100: BlockingRecv error: recv got EOF from 127.0.0.1:37100 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 16:39:06.473963 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.473300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.31.15.253:39029)
0809 16:39:06.473341 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.473343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.473506 (+ 163us) server_negotiation.cc:407] Connection header received
0809 16:39:06.473536 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.473536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.473546 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.473586 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.473910 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37106: BlockingRecv error: recv got EOF from 127.0.0.1:37106 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 16:39:06.507654 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.506925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37120 (local address 127.31.15.253:39029)
0809 16:39:06.506974 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.506976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.507107 (+ 131us) server_negotiation.cc:407] Connection header received
0809 16:39:06.507179 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.507180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.507192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.507242 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.507592 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37120: BlockingRecv error: recv got EOF from 127.0.0.1:37120 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 16:39:06.573480 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.572763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.31.15.253:39029)
0809 16:39:06.572859 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.572861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.572931 (+ 70us) server_negotiation.cc:407] Connection header received
0809 16:39:06.572995 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.572997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.573009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.573061 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.573418 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60070: BlockingRecv error: recv got EOF from 127.0.0.1:60070 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 16:39:06.703363 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.702640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.31.15.253:39029)
0809 16:39:06.702730 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.702733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.702805 (+ 72us) server_negotiation.cc:407] Connection header received
0809 16:39:06.702873 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.702874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.702886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.702936 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.703299 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60076: BlockingRecv error: recv got EOF from 127.0.0.1:60076 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 16:39:06.961102 3857 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 16:39:06.960494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.31.15.253:39029)
0809 16:39:06.960560 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:06.960561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:06.960588 (+ 27us) server_negotiation.cc:407] Connection header received
0809 16:39:06.960660 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:06.960661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:06.960671 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:06.960719 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 16:39:06.961040 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60078: BlockingRecv error: recv got EOF from 127.0.0.1:60078 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 16:39:07.323413 4021 catalog_manager.cc:1259] Loaded cluster ID: 43629c53828b4116963e1a90d9eadfb8
I20260809 16:39:07.323493 4021 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: loading cluster ID for follower catalog manager: success
I20260809 16:39:07.324267 4021 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: acquiring CA information for follower catalog manager: success
I20260809 16:39:07.324539 4021 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c57d7a7f448340d2b86de8b7cf0d8892: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:39:07.402938 4071 catalog_manager.cc:1259] Loaded cluster ID: 43629c53828b4116963e1a90d9eadfb8
I20260809 16:39:07.403034 4071 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: loading cluster ID for follower catalog manager: success
I20260809 16:39:07.403929 4071 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: acquiring CA information for follower catalog manager: success
I20260809 16:39:07.404307 4071 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14b4171caef44fd98d9c85e878fdd61d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 16:39:07.476578 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:07.474192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.31.15.253:39029)
0809 16:39:07.474306 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:07.474308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:07.474319 (+ 11us) server_negotiation.cc:407] Connection header received
0809 16:39:07.474386 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:07.474386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:07.474448 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:07.474519 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:07.474640 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.475136 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.475947 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.476151 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.476203 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:07.476312 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:07.476424 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:07.476478 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:07.476494 (+ 16us) server_negotiation.cc:299] Negotiation successful
0809 16:39:07.476510 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":53,"threads_started":1}
I20260809 16:39:07.480710 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:07.478277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36748 (local address 127.31.15.252:42397)
0809 16:39:07.478375 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:07.478377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:07.478389 (+ 12us) server_negotiation.cc:407] Connection header received
0809 16:39:07.478512 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:07.478513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:07.478549 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:07.478609 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:07.478704 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.479214 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.479971 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.480195 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.480244 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:07.480395 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:07.480500 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:07.480564 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:07.480602 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 16:39:07.480622 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":48,"threads_started":1}
I20260809 16:39:07.484943 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:07.482267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36758 (local address 127.31.15.252:42397)
0809 16:39:07.482308 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:07.482309 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:07.482649 (+ 340us) server_negotiation.cc:407] Connection header received
0809 16:39:07.482679 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:07.482680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:07.482692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:07.482739 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:07.482923 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.483615 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.484318 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.484558 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.484622 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:07.484717 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:07.484746 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:07.484793 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:07.484845 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 16:39:07.484865 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 16:39:07.485232 4190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:07.482236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.31.15.253:39029)
0809 16:39:07.482283 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:07.482285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:07.482476 (+ 191us) server_negotiation.cc:407] Connection header received
0809 16:39:07.482534 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:07.482535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:07.482547 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:07.482591 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:07.482723 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.483420 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.484488 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.484755 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.484812 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:07.484962 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:07.485001 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:07.485046 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:07.485134 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 16:39:07.485157 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 16:39:07.485809 4213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 16:39:07.482294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.31.15.254:46343)
0809 16:39:07.482415 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 16:39:07.482418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 16:39:07.482516 (+ 98us) server_negotiation.cc:407] Connection header received
0809 16:39:07.482602 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 16:39:07.482603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 16:39:07.482666 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 16:39:07.482742 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 16:39:07.482931 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.483456 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.484851 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 16:39:07.485146 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 16:39:07.485203 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 16:39:07.485474 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 16:39:07.485580 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 16:39:07.485640 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 16:39:07.485701 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 16:39:07.485728 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260809 16:39:07.487442 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3643
I20260809 16:39:07.502144 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3754
I20260809 16:39:07.509729 31807 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskfvbJG8/build/release/bin/kudu with pid 3860
2026-08-09T16:39:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2967 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5869 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (275123 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST